1 Commits

Author SHA1 Message Date
5c43740a04 fix(issue-7): AGENTS.md is absent in the repository 2026-07-01 09:02:44 +00:00

411
AGENTS.md
View File

@ -1,227 +1,250 @@
# AGENTS.md # AGENTS.md
Guidance for AI agents (and humans) working in this repository. Read this Guidance for AI agents (and humans) working in this repository.
before making changes.
## Project overview ## Project Overview
**kurious** (module `git.loyso.art/frx/kurious`) is a Go service that **kurious** (`git.loyso.art/frx/kurious`) is a course/education platform that
aggregates, stores, and serves educational/learning course data sourced from aggregates and serves educational course listings. It scrapes/syncs course data
[sravni.ru](https://www.sravni.ru/kursy). It exposes a server-rendered web UI from an external source (sravni.ru) via a rate-limited HTTP client, stores it
for browsing courses and runs a background job that periodically syncs data locally, and exposes it through a server-rendered web UI with filtering,
from the upstream sravni.ru API. pagination, and statistics.
Go version: **1.26** (`toolchain go1.26.4`). Builds are CGO-free The project is written in Go and follows a hexagonal (ports & adapters)
(`CGO_ENABLED=0`), using the pure-Go `modernc.org/sqlite` driver. architecture with a CQRS-flavored application layer.
### Entry points (`cmd/`) ## Tech Stack
| Binary | Path | Purpose | - **Language:** Go 1.26 (toolchain `go1.26.4`; see `go.mod`)
| -------------- | ----------------------- | ----------------------------------------------------------------------- | - **HTTP routing:** `github.com/gorilla/mux`
| `kuriousweb` | `cmd/kuriweb` | HTTP web server. Serves the course browser UI + REST endpoints. | - **HTTP client:** `github.com/go-resty/resty/v2` (sravni.ru scraper)
| `kuriousbg` | `cmd/background` | Background worker. Cron-scheduled sync of courses/orgs from sravni.ru. | - **Database:** SQLite via `modernc.org/sqlite` (pure-Go, CGO disabled).
| `sravnicli` | `cmd/dev/sravnicli` | Developer/debug CLI for inspecting the sravni.ru API and redux state. | YDB was historically supported but is **no longer supported**
(`service.NewApplication` returns an error for the YDB engine).
Each binary reads a JSON config from `argv[1]` (default `config.json` for - **DB access:** `github.com/jmoiron/sqlx` (named queries)
servers, `config_cli.json` for the CLI). Note: `*.json` is gitignored. - **Templating:** `github.com/a-h/templ` (`.templ` files compile to Go)
- **Observability:** OpenTelemetry (`go.opentelemetry.io/otel`) — traces and
Build metadata (`version`, `commit`, `buildTime`) is injected via `-ldflags` metrics with stdout / OTLP (HTTP & gRPC) exporters
into the root package `kurious.go` (see `Version()`, `Commit()`, `BuildTime()`). - **Background jobs:** `github.com/robfig/cron/v3`
- **Logging:** standard `log/slog` (text or JSON, configurable)
- **Rate limiting:** `golang.org/x/time/rate`
- **Testing:** `github.com/stretchr/testify` (assert, require, mock)
- **Mocking:** `github.com/vektra/mockery/v2` (config in `.mockery.yaml`)
- **Linting:** `golangci-lint` v1.55.2
- **Build/task runner:** [Taskfile](https://taskfile.dev) (`Taskfile.yml`)
- **Build flags:** ldflags inject `version`, `commit`, `buildTime`
(see `kurious.go`)
## Architecture ## Architecture
The codebase follows **Hexagonal Architecture (Ports & Adapters)** with The codebase implements a **hexagonal architecture** (ports & adapters) with a
**CQRS** (command/query separation). The domain core has no knowledge of CQRS-style separation between commands (writes) and queries (reads).
databases, HTTP, or external services; everything is wired together in the
composition root.
``` ```
cmd/ # application entrypoints (thin mains) ┌─────────────────────────────────────────────┐
internal/kurious/ delivery │ ports/ (HTTP server, cron jobs) │
domain/ # CORE: entities, repository interfaces, value/param types mechanisms └──────────────────────┬──────────────────────┘
app/ # application layer │ depends on
command/ # write-side handlers (CQRS commands) ┌──────────────────────▼──────────────────────┐
query/ # read-side handlers (CQRS queries) application app/ (command/, query/) │
app.go # Application struct aggregating Commands + Queries layer app.Application { Commands, Queries } │
ports/ # DRIVING adapters (what drives the domain) │ decorator/ (logging decorators) │
http/ # HTTP server (gorilla/mux + templ HTML) └──────────────────────┬──────────────────────┘
background/ # cron job handlers │ depends on
background.go # cron scheduler wrapper ┌──────────────────────▼──────────────────────┐
services.go # Services aggregate domain domain/ (entities, repository ports) │
adapters/ # DRIVEN adapters (what the domain drives) └──────────────────────┬──────────────────────┘
sqlite_* # SQLite repositories (course, organization, learning category) │ implemented by
memory_mapper.go # in-memory CourseMapper implementation ┌──────────────────────▼──────────────────────┐
ydb_course_repository.go # legacy YDB adapter (deprecated, unsupported) adapters │ adapters/ (sqlite_*, memory_mapper, │
mocks/ # generated mockery mocks │ ydb_* legacy stub) │
service/ # COMPOSITION ROOT: wires adapters into app.Application └─────────────────────────────────────────────┘
internal/common/ # cross-cutting utilities (see below)
pkg/xdefault/ # reusable helper (WithFallback)
migrations/sqlite/ # numbered *.sql migrations + embed migrator
assets/kurious/ # embedded static assets (go:embed)
``` ```
### Layer responsibilities - **`domain/`** — pure business entities (`Course`, `Organization`,
`LearningCategory`) and the **repository interfaces** (ports) that the
application depends on: `CourseRepository`, `OrganizationRepository`,
`LearningCategoryRepository`. No I/O or framework code lives here.
- **`app/`** — the application layer. `app.Application` aggregates a
`Commands` struct and a `Queries` struct. Handlers in `app/command/` perform
writes; handlers in `app/query/` perform reads. Each handler implements a
generic `decorator.CommandHandler[T]` or `decorator.QueryHandler[Q, U]`
interface and is wrapped with logging decorators at construction time.
- **`adapters/`** — concrete implementations of the domain repository ports.
`sqlite_course_repository.go`, `sqlite_organization_repository.go`, etc.
translate between domain types and SQL rows. `memory_mapper.go` is an
in-memory mapper translating external dictionary IDs to human-readable names.
- **`ports/`** — delivery mechanisms. `ports/http/` is the HTTP server (gorilla/mux
+ templ templates); `ports/background/` runs scheduled cron jobs (e.g.
`SyncSravniHandler`).
- **`service/`** — the **composition root**. `service.NewApplication` selects
the DB engine, constructs the SQLite connection, builds the adapters, and
wires everything into an `app.Application`. This is what binaries call.
- **`domain/`** — Pure business types: `Course`, `Organization`, Data flows: an HTTP request enters `ports/http`, calls a handler on
`LearningCategory`, `Category`. Defines repository interfaces `service.Application` (which delegates to `app/command` or `app/query`), which
(`CourseRepository`, `OrganizationRepository`, `LearningCategoryRepository`) calls a `domain` repository interface implemented by an `adapters/*` repository.
and the `CourseMapper` interface. Each repository interface carries a
`//go:generate mockery ...` directive. Params/results are plain structs.
- **`app/`** — Use-case handlers. `app.go` defines `Application{ Commands,
Queries }`. Commands (`app/command/`) mutate state; queries (`app/query/`)
read and project. Each handler is a private struct exposed via a
`NewXxxHandler` constructor that returns a decorated
`decorator.CommandHandler[T]` / `decorator.QueryHandler[Q, U]`.
- **`adapters/`** — Implementations of `domain` interfaces. SQLite is the
active engine; `sqlite_connection.go` opens the DB and runs migrations.
`memory_mapper.go` holds dictionary name/id maps and aggregate counts.
- **`ports/`** — Inbound transports. `ports/http/server.go` renders with templ
templates and normalizes errors to HTTP status codes. `ports/background.go`
wraps `robfig/cron/v3`.
- **`service/`** — `NewApplication` is the composition root: picks the DB
engine, builds repositories, and constructs all commands/queries.
### `internal/common/` utility packages ## Directory Structure
| Package | Role | ```
| ------------ | ------------------------------------------------------------------------------------ | .
| `config` | JSON config structs (`Log`, `HTTP`, `Sqlite`, `YDB`, `Trace`) + `NewSLogger` factory | ├── kurious.go # Root package: version/commit/buildTime getters
| `errors` | Sentinel errors (`ErrNotFound`, `ErrNotImplemented`, `ErrUnexpectedStatus` as `SimpleError`) and `ValidationError` | ├── Taskfile.yml # Build, test, lint, generate, run task definitions
| `decorator` | CQRS decorators adding slog logging + OpenTelemetry tracing around every handler | ├── .mockery.yaml # mockery config (with-expecter, keeptree)
| `xcontext` | Context-aware structured logging (`LogInfo/Debug/Error/...`) carrying request id + log fields | ├── go.mod / go.sum
| `xlog` | Log level/format enums + slog adapters (e.g. cron logger bridge) | ├── cmd/ # Entry points (one package per binary)
| `nullable` | Generic `nullable.Value[T]` for optional fields | │ ├── kuriweb/ # Main HTTP web server (config.go, main.go, http.go, trace.go)
| `xslices` | Generic slice helpers (`Map`, ...) | │ ├── background/ # Background sync process (cron-driven sravni sync)
| `generator` | ID generation (e.g. `RandomInt64ID` for request ids) | │ └── dev/sravnicli/ # Developer CLI for inspecting the sravni source
| `client/sravni` | HTTP client for sravni.ru (resty), parses `__NEXT_DATA__` redux state, rate-limited | ├── internal/
| `xdefault` (pkg) | `WithFallback(value, default)` helper | │ ├── kurious/ # Application core (the hexagon)
│ │ ├── domain/ # Entities + repository interface ports
│ │ ├── app/
│ │ │ ├── app.go # Application{Commands, Queries} aggregate
│ │ │ ├── command/ # Write-side handlers (Create, Delete, Update...)
│ │ │ └── query/ # Read-side handlers (List, Get, Stats...)
│ │ ├── adapters/ # Repository implementations + mocks/
│ │ ├── ports/
│ │ │ ├── http/ # HTTP server, course routes
│ │ │ │ └── bootstrap/ # templ templates (.templ) + generated _templ.go
│ │ │ ├── background/ # Cron job handlers
│ │ │ ├── background.go # BackgroundProcess (cron scheduler)
│ │ │ └── services.go # Services{HTTP, Background} aggregate
│ │ └── service/ # Composition root: NewApplication wiring
│ └── common/ # Shared, reusable utilities
│ ├── client/sravni/ # sravni.ru HTTP client + mocks/
│ ├── config/ # Config structs (HTTP, Log, Sqlite, Trace, YDB, Duration)
│ ├── decorator/ # Command/Query handler interfaces + logging decorators
│ ├── errors/ # SimpleError, ValidationError, sentinel errors
│ ├── generator/ # ID generators
│ ├── nullable/ # Generic nullable Value[T]
│ ├── xcontext/ # context helpers (request id, log fields)
│ ├── xlog/ # slog + cron logger adapters
│ └── xslices/ # slice helpers (Map, Filter, ForEach, LRU)
├── pkg/
│ └── xdefault/ # WithFallback helper (public-ish pkg)
├── migrations/
│ └── sqlite/ # SQL migrations (embed.FS) + migrator.go
├── assets/kurious/static/ # Static web assets (embedded via go:embed)
└── htmlexamples/ # Standalone HTML/templ prototyping examples
```
## Build & test commands ## Build, Test, and Lint Commands
The project uses [Task](https://taskfile.dev) (`Taskfile.yml`). Install tools All operations are driven by **Taskfile** (`task <name>`). The toolchain is
first with `task install_tools` (installs `golangci-lint`, `templ`, `mockery` installed into a local `bin/` (`GOBIN={{.USER_WORKING_DIR}}/bin`) and
into `./bin`). `CGO_ENABLED=0` is enforced.
| Task | What it does | | Command | What it does |
| ----------------------- | ------------------------------------------------------------------ | |--------------------------|--------------------------------------------------------------------|
| `task install_tools` | Install dev tools into `./bin` | | `task install_tools` | Install `golangci-lint`, `templ`, `mockery` into `bin/` |
| `task generate` | Run `templ generate` for `*.templ` sources (run before build/test) | | `task generate` | Run `templ generate` (compiles `.templ``_templ.go`) |
| `task mocks` | `go generate ./internal/...` (regenerate mockery mocks) | | `task mocks` | Run `go generate ./internal/...` (regenerate mockery mocks) |
| `task check` | `golangci-lint run ./...` (depends on `generate`) | | `task check` | Run `golangci-lint run ./...` (depends on `generate`) |
| `task test` | `go test ./internal/...` (depends on `generate`) | | `task test` | Run `go test ./internal/...` (depends on `generate`) |
| `task build_web` | Build `kuriousweb` (depends on `check`, `test`) | | `task build_web` | Build `bin/kuriousweb` (depends on `check` + `test`) |
| `task build_background` | Build `kuriousbg` (depends on `check`, `test`) | | `task build_background` | Build `bin/kuriousbg` (depends on `check` + `test`) |
| `task build_dev_cli` | Build `sravnicli` (depends on `check`, `test`) | | `task build_dev_cli` | Build `bin/sravnicli` (depends on `check` + `test`) |
| `task build` | Build all three binaries | | `task build` | Build all three binaries |
| `task run` | Build then run `kuriousweb` | | `task run` | Build then run `bin/kuriousweb` |
Plain Go equivalents (use when Task is unavailable): Typical workflow before committing: `task check && task test`.
### Running the binaries
Binaries read a JSON config file path as their first argument, defaulting to
`config.json` (note: `*.json` is gitignored, so configs are local). Example
fields: `log`, `sqlite`, `http.listen_addr`, `db_engine`, `tracing`.
```bash ```bash
go generate ./internal/... # regenerate mocks ./bin/kuriousweb path/to/config.json
templ generate # render .templ -> .go
golangci-lint run ./... # lint
go test ./... # run all tests
go build -o bin/kuriousweb ./cmd/kuriweb # build a binary
``` ```
> Note: `templ generate` must run before `go build`/`go test` whenever a ## Code Conventions
> `*.templ` file changes — the generated `*_templ.go` files are committed-free
> and required for compilation.
## Code conventions ### Decorators / handler pattern
Command and query handlers implement generic interfaces and are wrapped with
logging decorators at construction time. When adding a new use case:
1. Define the command/query struct and a `*Handler` type alias of
`decorator.CommandHandler[T]` / `decorator.QueryHandler[Q, U]` in
`app/command/` or `app/query/`.
2. Implement a private `*Handler` struct holding its repository dependencies.
3. Expose a `New*Handler(deps..., log *slog.Logger)` constructor that returns
the decorated handler (`decorator.ApplyCommandDecorators` /
`decorator.AddQueryDecorators`).
4. Register the new handler on `app.Commands` / `app.Queries` in
`app/app.go` and wire it in `service/service.go`.
### Mock generation (mockery)
- Interfaces marked with `//go:generate mockery ...` directives (see
`domain/repository.go`, `common/client/sravni/client.go`) are mocked into
sibling `mocks/` packages (`keeptree: True`, `with-expecter: true`).
- Regenerate with `task mocks` (runs `go generate ./internal/...`).
- Mocks are committed to the repo.
### Templating (templ)
- HTML is authored as `.templ` files under
`internal/kurious/ports/http/bootstrap/`.
- `task generate` compiles them to `*_templ.go` (committed). Run it whenever a
`.templ` file changes; `task check` and `task test` both depend on it.
### Database / migrations
- SQLite migrations live in `migrations/sqlite/*.sql` and are embedded via
`go:embed`. The migrator (`migrations/sqlite/migrator.go`) applies them in
order inside a transaction.
- Repository adapters use `sqlx` named queries; domain ↔ row translation is
centralized via `AsDomain()` methods.
### Logging & observability
- Use `log/slog` with structured attributes. Request-scoped fields (e.g.
`request_id`) are propagated through `context` via `internal/common/xcontext`.
- HTTP handlers wrap requests in trace spans and record metrics
(request duration histogram). Database adapters emit spans tagged with
`db.*` attributes.
### Error handling ### Error handling
- Wrap errors with context using `fmt.Errorf("<action>: %w", err)`. Preserve - Domain/repository errors use sentinels from `internal/common/errors`
the chain so callers can `errors.Is` / `errors.As`. (`ErrNotFound`, `ErrNotImplemented`) and `*ValidationError` (mapped to HTTP
- Use the sentinels in `internal/common/errors/error.go`: 400/404 in `ports/http/server.go`).
- `errors.ErrNotFound` / `errors.ErrNotImplemented` / `errors.ErrUnexpectedStatus` - Wrap errors with `fmt.Errorf("doing X: %w", err)` to add context while
(these are `SimpleError`, a string-backed error type). preserving the underlying error for `errors.Is` / `errors.As`.
- `errors.NewValidationError(field, reason)` for invalid input.
- The HTTP layer (`ports/http/server.go::handleError`) maps these to status
codes: `ValidationError` → 400, `ErrNotFound` → 404, anything else → 500
(with a generic message, never leaking internals).
- Repository `Get` methods contractually return `ErrNotFound` when a row is
missing — honor this in new adapters.
### Logging ## Testing Patterns
- Use `log/slog` (structured), never `fmt.Println` in library code. - **Scope:** unit tests live next to the code they test
- Prefer the `internal/common/xcontext` helpers (`LogInfo`, `LogDebug`, (`*_test.go`). `task test` runs `go test ./internal/...`.
`LogError`, `LogWithError`, `LogWithWarnError`) over raw `log.InfoContext`, - **Framework:** `github.com/stretchr/testify``require` for fatal
because they automatically attach context-scoped fields (e.g. `request_id`, assertions, `assert` for non-fatal, `mock` for mock interactions.
`handler`). - **Mocks:** generated mockery mocks with the **expecter** API:
- Create loggers via `config.NewSLogger(cfg.Log)` and tag components with ```go
`log.With(slog.String("component", "..."))`. repo.EXPECT().Create(mock.Anything, expectedParams).
- In tests, use a discard logger: `slog.New(slog.NewTextHandler(io.Discard, nil))`. Return(domain.Course{ID: "c1"}, nil).Once()
```
Note: when the generated mock does not yet implement every interface method,
tests embed the mock and add the missing methods (see the `fullRepo` pattern
in `internal/kurious/app/command/command_test.go`).
- **Logger:** tests use a `quietLogger()` helper that discards output
(`slog.New(slog.NewTextHandler(io.Discard, nil))`).
- **Adapter tests:** `internal/kurious/adapters/sqlite_*_test.go` exercise the
real SQLite repositories against an in-memory/temp database.
- **Conventions:** table-driven where appropriate; use `context.Background()`
in tests; assert on wrapped errors with `assert.ErrorIs` and message
fragments with `assert.Contains`.
### CQRS & handlers ## Notes for Agents
- Every command/query handler is wrapped at construction time by - Run `task generate` before linting/testing if you touched any `.templ` file.
`decorator.ApplyCommandDecorators` / `decorator.AddQueryDecorators`. The - Do not re-introduce YDB as a working engine without updating
decorator adds a tracing span and start/finish log lines — **do not** add `service.NewApplication` and the migration story (currently SQLite-only).
your own top-level logging inside `Handle`; rely on the decorator. - Generated files (`*_templ.go`, `mocks/*.go`) are committed — regenerate and
- Handler signature pattern: commit them alongside source changes.
- command: `Handle(ctx context.Context, cmd T) error` - `*.json`, `*.sqlite`, `bin/`, and `*.log` are gitignored; do not commit
- query: `Handle(ctx context.Context, q Q) (U, error)` local configs or databases.
- Keep handlers thin: translate the command/query into domain params, call the
repository, and map results. Put business types in `domain/`.
### Naming & structure
- Constructors are `NewXxx`. Unexported handler struct, exported aliased type
(`decorator.CommandHandler[T]` / `decorator.QueryHandler[Q, U]`).
- Nullable fields use `nullable.Value[T]` rather than pointers.
- Exported interfaces that external code might mock get a
`//go:generate mockery --name <Interface> --output ../adapters/mocks` line
directly above the interface declaration.
### Tracing & metrics
- OpenTelemetry is wired throughout (`cmd/*/trace.go` sets up the SDK).
- DB adapters use the `adapters.db` tracer; CQRS decorators use the `api`
tracer; the HTTP layer uses the `web` tracer. Record errors on spans with
`span.RecordError(err)` and set `codes.Error` status.
## Testing
- Framework: `github.com/stretchr/testify` (`suite`, `assert`, `require`,
`mock`). Mocks are generated by mockery into `internal/kurious/adapters/mocks`
and `internal/common/client/sravni/mocks`.
- SQLite adapter tests (`internal/kurious/adapters/*_test.go`) extend
`sqliteBaseSuite`, which spins up an in-memory (`:memory:`) SQLite DB and
applies the real migrations. `TearDownTest` clears tables between cases.
- Command/query tests use mockery mocks and a discard logger.
- After changing interfaces, regenerate mocks: `go generate ./internal/...`
(or `task mocks`).
## Key dependencies
| Dependency | Role |
| --------------------------------- | ------------------------------------------------------------- |
| `github.com/gorilla/mux` | HTTP routing & middleware |
| `github.com/a-h/templ` | Type-safe HTML templates (`*.templ`) |
| `github.com/jmoiron/sqlx` | Ergonomic SQL layer over `database/sql` |
| `modernc.org/sqlite` | Pure-Go (CGO-free) SQLite driver |
| `github.com/robfig/cron/v3` | Cron scheduler for the background sync |
| `github.com/go-resty/resty/v2` | HTTP client for the sravni.ru API |
| `golang.org/x/net/html` | HTML parsing of sravni.ru `__NEXT_DATA__` redux state |
| `golang.org/x/time/rate` | Rate limiting the sravni.ru client |
| `golang.org/x/sync` | `errgroup` for goroutine lifecycle |
| `github.com/teris-io/cli` | CLI framework for `sravnicli` |
| `go.opentelemetry.io/otel` (+ exporters) | Tracing & metrics (OTLP gRPC/HTTP, stdout) |
| `github.com/stretchr/testify` | Test assertions, suites, and mocks |
| `github.com/ydb-platform/ydb-go-sdk/v3` | YDB SDK — **deprecated**, `NewApplication` returns an error for the `ydb` engine |
## Gotchas
- `*.json` is gitignored (configs contain local settings). Do not commit
config files; document config shape in `cmd/<binary>/config.go` instead.
- The `background` binary currently panics on start
(`savingOrganizationIDInternalInsteadOfExternal` guard) until the
organization-id saving semantics are fixed.
- Templ-generated files are not committed; always run `templ generate` (or
`task generate`) after pulling or before building/testing.
- The YDB engine path is dead code kept for reference — do not extend it; add
new storage work to the SQLite adapters.