LinkLab — Week 5 Blueprint

CS4297 Applied System Design · CI/CD with Forgejo Actions + ElasticSearch

Complete Week 4 solution included. Clone this and start here.

---

What you are building this week

Part	What happens	Time
A	Push to Forgejo, set up Actions pipeline, trigger first green build	30 min
B	Implement ElasticSearch indexing and search endpoint	50 min
C	Observe the fan-out pattern across all four datastores	15 min
Optional 1	Multi-field search with custom ES query and fuzziness	30 min
Optional 2	Pipeline notification — post PR comment after deploy	30 min
Optional 3	Re-index endpoint — rebuild ES index from MySQL	30 min

---

Prerequisites

• IntelliJ IDEA Ultimate — free at jetbrains.com/student
• JDK 21 — check with java -version
• Docker Desktop with at least 6 GB RAM (Settings → Resources → Memory)
• Forgejo account — register at cs4297.skynet.ie

---

Getting started

1. Start the infrastructure

This checkout uses Compose project cs4297-week-05 and container names prefixed with linklab-week5- so it can run alongside other projects. Its host ports are:

Service	Host port
MySQL	3307
Redis	6385
DynamoDB Local	8005
DynamoDB Admin	8006
ElasticSearch	9205
Kibana	5605
Application (IntelliJ / bootRun)	8085
NGINX (containerised application)	8086
Prometheus	9095
Grafana	3005

The application properties and Postman collection use these ports. Containers still communicate using service names and their original internal ports.

docker compose up -d mysql redis dynamodb-local dynamodb-admin elasticsearch kibana

ElasticSearch takes ~30 seconds to become healthy. Kibana takes another ~30 seconds after that.

docker compose ps   # all should show "healthy" or "running"

2. Run the application

From IntelliJ: open LinkLabApplication.kt and click ▶

From terminal (Mac/Linux): ./gradlew bootRun From terminal (Windows): .\gradlew.bat bootRun

3. Run the tests

# Mac/Linux
./gradlew test

# Windows
.\gradlew.bat test

Before you've written any code, this will fail to compile — not just fail tests — with errors like Unresolved reference 'findByOriginalUrlContaining'. That's expected: LinkSearchServiceTest already calls the method you add in Part B step 1, and Kotlin compiles all test files together, so one missing method blocks the whole test task. Add the one-line method signature to LinkSearchRepository first (Part B step 1), then re-run — you should see:

• LinkServiceTest, LinkControllerTest, CacheTest, ClickEventServiceTest, GlobalExceptionHandlerTest — all pass (Week 4 complete)
• LinkSearchServiceTest — 5 tests fail until you implement the rest of Part B. Expected.
• SearchControllerTest — 4 tests fail until you implement SearchController.search() and reindex(). Expected.

4. Open Kibana

Visit http://localhost:5605

Go to Dev Tools (wrench icon, left sidebar) and run:

GET /_cluster/health

You should see "status": "green" or "yellow" (yellow is fine for single-node).

---

What to implement

src/main/kotlin/ie/ul/cs4297/linklab/
├── search/
│   ├── LinkDocument.kt              # Already complete — read it
│   ├── LinkSearchRepository.kt      # PART B: add findByOriginalUrlContaining()
│   └── LinkSearchService.kt         # PART B: implement index() and search()
└── controller/
    └── SearchController.kt          # PART B: implement GET /api/links/search?q=

src/main/kotlin/ie/ul/cs4297/linklab/service/
└── LinkService.kt                   # PART B: uncomment linkSearchService.index() call

Implement in this order:

1. LinkSearchRepository — add the derived query method
2. LinkSearchService.index() — save to ES, never throw
3. LinkSearchService.search() — query ES, return empty list if unavailable
4. SearchController.search() — validate input, return results
5. LinkService.shorten() — uncomment the ES index call

Everything else is the complete Week 4 solution. Do not modify it.

---

Setting up the Forgejo Actions pipeline (Part A)

1. Create your Forgejo repository

Log in to cs4297.skynet.ie and create a new repo named cs4297-linklab.

2. Push this blueprint to Forgejo

git remote set-url origin https://cs4297.skynet.ie/<your-username>/cs4297-linklab.git
git push -u origin main

3. Add a secret for the container registry

• Forgejo → avatar → Settings → Applications → generate token (ci-token, scope: write:package)
• Repository → Settings → Secrets → Actions → New secret
• Name: ACCESS_TOKEN, value: your token

4. Trigger the pipeline

Create a branch, commit, push, open a PR on Forgejo, merge it. Watch the pipeline run under repository → Actions tab.

---

Fan-out pattern — four datastores

After implementing Part B, POST /api/links writes to:

Store	Purpose	Fails gracefully?
MySQL	Source of truth	No
Redis cache	Fast redirects	Yes
Redis leaderboard	Click ranking	Yes
ElasticSearch	Full-text search	Yes

Week 6 (Kafka) makes the ES and leaderboard writes asynchronous.

---

Week-by-week progression

Week	Content	Blueprint
2	REST API + MySQL	Stubs for LINK-001
3	Docker · Redis · NGINX · Grafana	Complete Week 2
4	SQL vs NoSQL · DynamoDB	Complete Week 3
5	CI/CD · ElasticSearch · Fan-out	← you are here
6	Kafka async pipeline	Complete Week 5
7	JWT + OAuth2	Complete Week 6
8	Kubernetes	Complete Week 7
9	Capstone: full integration	Complete Week 8

---

Common issues

ElasticSearch crashes on startup Docker Desktop doesn't have enough memory. Increase to at least 6 GB in Settings → Resources → Memory.

ConnectException to ElasticSearch on bootRun ES is not yet healthy. Run docker compose ps and wait until ES shows healthy.

LinkSearchServiceTest fails with UnsatisfiedDependencyException The ES auto-config is not excluded in the test profile. Check application-test.properties has the five spring.autoconfigure.exclude entries.

Pipeline fails at the Docker push step ACCESS_TOKEN secret is missing or has wrong scope. It needs write:package.

Kibana shows "Kibana server is not ready yet" Wait another 30 seconds — Kibana waits for ES to be fully ready before starting.

./gradlew bootRun fails with java.lang.IllegalArgumentException Gradle is using JDK 25. Run .\gradlew --stop then set Gradle JVM to Java 21 in IntelliJ Settings → Build Tools → Gradle → Gradle JVM.