Compare commits
2 Commits
fix/sravni
...
8566b98b3b
| Author | SHA1 | Date | |
|---|---|---|---|
| 8566b98b3b | |||
| 228b30ccf4 |
227
AGENTS.md
Normal file
227
AGENTS.md
Normal file
@ -0,0 +1,227 @@
|
||||
# AGENTS.md
|
||||
|
||||
Guidance for AI agents (and humans) working in this repository. Read this
|
||||
before making changes.
|
||||
|
||||
## Project overview
|
||||
|
||||
**kurious** (module `git.loyso.art/frx/kurious`) is a Go service that
|
||||
aggregates, stores, and serves educational/learning course data sourced from
|
||||
[sravni.ru](https://www.sravni.ru/kursy). It exposes a server-rendered web UI
|
||||
for browsing courses and runs a background job that periodically syncs data
|
||||
from the upstream sravni.ru API.
|
||||
|
||||
Go version: **1.26** (`toolchain go1.26.4`). Builds are CGO-free
|
||||
(`CGO_ENABLED=0`), using the pure-Go `modernc.org/sqlite` driver.
|
||||
|
||||
### Entry points (`cmd/`)
|
||||
|
||||
| Binary | Path | Purpose |
|
||||
| -------------- | ----------------------- | ----------------------------------------------------------------------- |
|
||||
| `kuriousweb` | `cmd/kuriweb` | HTTP web server. Serves the course browser UI + REST endpoints. |
|
||||
| `kuriousbg` | `cmd/background` | Background worker. Cron-scheduled sync of courses/orgs from sravni.ru. |
|
||||
| `sravnicli` | `cmd/dev/sravnicli` | Developer/debug CLI for inspecting the sravni.ru API and redux state. |
|
||||
|
||||
Each binary reads a JSON config from `argv[1]` (default `config.json` for
|
||||
servers, `config_cli.json` for the CLI). Note: `*.json` is gitignored.
|
||||
|
||||
Build metadata (`version`, `commit`, `buildTime`) is injected via `-ldflags`
|
||||
into the root package `kurious.go` (see `Version()`, `Commit()`, `BuildTime()`).
|
||||
|
||||
## Architecture
|
||||
|
||||
The codebase follows **Hexagonal Architecture (Ports & Adapters)** with
|
||||
**CQRS** (command/query separation). The domain core has no knowledge of
|
||||
databases, HTTP, or external services; everything is wired together in the
|
||||
composition root.
|
||||
|
||||
```
|
||||
cmd/ # application entrypoints (thin mains)
|
||||
internal/kurious/
|
||||
domain/ # CORE: entities, repository interfaces, value/param types
|
||||
app/ # application layer
|
||||
command/ # write-side handlers (CQRS commands)
|
||||
query/ # read-side handlers (CQRS queries)
|
||||
app.go # Application struct aggregating Commands + Queries
|
||||
ports/ # DRIVING adapters (what drives the domain)
|
||||
http/ # HTTP server (gorilla/mux + templ HTML)
|
||||
background/ # cron job handlers
|
||||
background.go # cron scheduler wrapper
|
||||
services.go # Services aggregate
|
||||
adapters/ # DRIVEN adapters (what the domain drives)
|
||||
sqlite_* # SQLite repositories (course, organization, learning category)
|
||||
memory_mapper.go # in-memory CourseMapper implementation
|
||||
ydb_course_repository.go # legacy YDB adapter (deprecated, unsupported)
|
||||
mocks/ # generated mockery mocks
|
||||
service/ # COMPOSITION ROOT: wires adapters into app.Application
|
||||
internal/common/ # cross-cutting utilities (see below)
|
||||
pkg/xdefault/ # reusable helper (WithFallback)
|
||||
migrations/sqlite/ # numbered *.sql migrations + embed migrator
|
||||
assets/kurious/ # embedded static assets (go:embed)
|
||||
```
|
||||
|
||||
### Layer responsibilities
|
||||
|
||||
- **`domain/`** — Pure business types: `Course`, `Organization`,
|
||||
`LearningCategory`, `Category`. Defines repository interfaces
|
||||
(`CourseRepository`, `OrganizationRepository`, `LearningCategoryRepository`)
|
||||
and the `CourseMapper` interface. Each repository interface carries a
|
||||
`//go:generate mockery ...` directive. Params/results are plain structs.
|
||||
- **`app/`** — Use-case handlers. `app.go` defines `Application{ Commands,
|
||||
Queries }`. Commands (`app/command/`) mutate state; queries (`app/query/`)
|
||||
read and project. Each handler is a private struct exposed via a
|
||||
`NewXxxHandler` constructor that returns a decorated
|
||||
`decorator.CommandHandler[T]` / `decorator.QueryHandler[Q, U]`.
|
||||
- **`adapters/`** — Implementations of `domain` interfaces. SQLite is the
|
||||
active engine; `sqlite_connection.go` opens the DB and runs migrations.
|
||||
`memory_mapper.go` holds dictionary name/id maps and aggregate counts.
|
||||
- **`ports/`** — Inbound transports. `ports/http/server.go` renders with templ
|
||||
templates and normalizes errors to HTTP status codes. `ports/background.go`
|
||||
wraps `robfig/cron/v3`.
|
||||
- **`service/`** — `NewApplication` is the composition root: picks the DB
|
||||
engine, builds repositories, and constructs all commands/queries.
|
||||
|
||||
### `internal/common/` utility packages
|
||||
|
||||
| Package | Role |
|
||||
| ------------ | ------------------------------------------------------------------------------------ |
|
||||
| `config` | JSON config structs (`Log`, `HTTP`, `Sqlite`, `YDB`, `Trace`) + `NewSLogger` factory |
|
||||
| `errors` | Sentinel errors (`ErrNotFound`, `ErrNotImplemented`, `ErrUnexpectedStatus` as `SimpleError`) and `ValidationError` |
|
||||
| `decorator` | CQRS decorators adding slog logging + OpenTelemetry tracing around every handler |
|
||||
| `xcontext` | Context-aware structured logging (`LogInfo/Debug/Error/...`) carrying request id + log fields |
|
||||
| `xlog` | Log level/format enums + slog adapters (e.g. cron logger bridge) |
|
||||
| `nullable` | Generic `nullable.Value[T]` for optional fields |
|
||||
| `xslices` | Generic slice helpers (`Map`, ...) |
|
||||
| `generator` | ID generation (e.g. `RandomInt64ID` for request ids) |
|
||||
| `client/sravni` | HTTP client for sravni.ru (resty), parses `__NEXT_DATA__` redux state, rate-limited |
|
||||
| `xdefault` (pkg) | `WithFallback(value, default)` helper |
|
||||
|
||||
## Build & test commands
|
||||
|
||||
The project uses [Task](https://taskfile.dev) (`Taskfile.yml`). Install tools
|
||||
first with `task install_tools` (installs `golangci-lint`, `templ`, `mockery`
|
||||
into `./bin`).
|
||||
|
||||
| Task | What it does |
|
||||
| ----------------------- | ------------------------------------------------------------------ |
|
||||
| `task install_tools` | Install dev tools into `./bin` |
|
||||
| `task generate` | Run `templ generate` for `*.templ` sources (run before build/test) |
|
||||
| `task mocks` | `go generate ./internal/...` (regenerate mockery mocks) |
|
||||
| `task check` | `golangci-lint run ./...` (depends on `generate`) |
|
||||
| `task test` | `go test ./internal/...` (depends on `generate`) |
|
||||
| `task build_web` | Build `kuriousweb` (depends on `check`, `test`) |
|
||||
| `task build_background` | Build `kuriousbg` (depends on `check`, `test`) |
|
||||
| `task build_dev_cli` | Build `sravnicli` (depends on `check`, `test`) |
|
||||
| `task build` | Build all three binaries |
|
||||
| `task run` | Build then run `kuriousweb` |
|
||||
|
||||
Plain Go equivalents (use when Task is unavailable):
|
||||
|
||||
```bash
|
||||
go generate ./internal/... # regenerate mocks
|
||||
templ generate # render .templ -> .go
|
||||
golangci-lint run ./... # lint
|
||||
go test ./... # run all tests
|
||||
go build -o bin/kuriousweb ./cmd/kuriweb # build a binary
|
||||
```
|
||||
|
||||
> Note: `templ generate` must run before `go build`/`go test` whenever a
|
||||
> `*.templ` file changes — the generated `*_templ.go` files are committed-free
|
||||
> and required for compilation.
|
||||
|
||||
## Code conventions
|
||||
|
||||
### Error handling
|
||||
|
||||
- Wrap errors with context using `fmt.Errorf("<action>: %w", err)`. Preserve
|
||||
the chain so callers can `errors.Is` / `errors.As`.
|
||||
- Use the sentinels in `internal/common/errors/error.go`:
|
||||
- `errors.ErrNotFound` / `errors.ErrNotImplemented` / `errors.ErrUnexpectedStatus`
|
||||
(these are `SimpleError`, a string-backed error type).
|
||||
- `errors.NewValidationError(field, reason)` for invalid input.
|
||||
- The HTTP layer (`ports/http/server.go::handleError`) maps these to status
|
||||
codes: `ValidationError` → 400, `ErrNotFound` → 404, anything else → 500
|
||||
(with a generic message, never leaking internals).
|
||||
- Repository `Get` methods contractually return `ErrNotFound` when a row is
|
||||
missing — honor this in new adapters.
|
||||
|
||||
### Logging
|
||||
|
||||
- Use `log/slog` (structured), never `fmt.Println` in library code.
|
||||
- Prefer the `internal/common/xcontext` helpers (`LogInfo`, `LogDebug`,
|
||||
`LogError`, `LogWithError`, `LogWithWarnError`) over raw `log.InfoContext`,
|
||||
because they automatically attach context-scoped fields (e.g. `request_id`,
|
||||
`handler`).
|
||||
- Create loggers via `config.NewSLogger(cfg.Log)` and tag components with
|
||||
`log.With(slog.String("component", "..."))`.
|
||||
- In tests, use a discard logger: `slog.New(slog.NewTextHandler(io.Discard, nil))`.
|
||||
|
||||
### CQRS & handlers
|
||||
|
||||
- Every command/query handler is wrapped at construction time by
|
||||
`decorator.ApplyCommandDecorators` / `decorator.AddQueryDecorators`. The
|
||||
decorator adds a tracing span and start/finish log lines — **do not** add
|
||||
your own top-level logging inside `Handle`; rely on the decorator.
|
||||
- Handler signature pattern:
|
||||
- command: `Handle(ctx context.Context, cmd T) error`
|
||||
- query: `Handle(ctx context.Context, q Q) (U, error)`
|
||||
- Keep handlers thin: translate the command/query into domain params, call the
|
||||
repository, and map results. Put business types in `domain/`.
|
||||
|
||||
### Naming & structure
|
||||
|
||||
- Constructors are `NewXxx`. Unexported handler struct, exported aliased type
|
||||
(`decorator.CommandHandler[T]` / `decorator.QueryHandler[Q, U]`).
|
||||
- Nullable fields use `nullable.Value[T]` rather than pointers.
|
||||
- Exported interfaces that external code might mock get a
|
||||
`//go:generate mockery --name <Interface> --output ../adapters/mocks` line
|
||||
directly above the interface declaration.
|
||||
|
||||
### Tracing & metrics
|
||||
|
||||
- OpenTelemetry is wired throughout (`cmd/*/trace.go` sets up the SDK).
|
||||
- DB adapters use the `adapters.db` tracer; CQRS decorators use the `api`
|
||||
tracer; the HTTP layer uses the `web` tracer. Record errors on spans with
|
||||
`span.RecordError(err)` and set `codes.Error` status.
|
||||
|
||||
## Testing
|
||||
|
||||
- Framework: `github.com/stretchr/testify` (`suite`, `assert`, `require`,
|
||||
`mock`). Mocks are generated by mockery into `internal/kurious/adapters/mocks`
|
||||
and `internal/common/client/sravni/mocks`.
|
||||
- SQLite adapter tests (`internal/kurious/adapters/*_test.go`) extend
|
||||
`sqliteBaseSuite`, which spins up an in-memory (`:memory:`) SQLite DB and
|
||||
applies the real migrations. `TearDownTest` clears tables between cases.
|
||||
- Command/query tests use mockery mocks and a discard logger.
|
||||
- After changing interfaces, regenerate mocks: `go generate ./internal/...`
|
||||
(or `task mocks`).
|
||||
|
||||
## Key dependencies
|
||||
|
||||
| Dependency | Role |
|
||||
| --------------------------------- | ------------------------------------------------------------- |
|
||||
| `github.com/gorilla/mux` | HTTP routing & middleware |
|
||||
| `github.com/a-h/templ` | Type-safe HTML templates (`*.templ`) |
|
||||
| `github.com/jmoiron/sqlx` | Ergonomic SQL layer over `database/sql` |
|
||||
| `modernc.org/sqlite` | Pure-Go (CGO-free) SQLite driver |
|
||||
| `github.com/robfig/cron/v3` | Cron scheduler for the background sync |
|
||||
| `github.com/go-resty/resty/v2` | HTTP client for the sravni.ru API |
|
||||
| `golang.org/x/net/html` | HTML parsing of sravni.ru `__NEXT_DATA__` redux state |
|
||||
| `golang.org/x/time/rate` | Rate limiting the sravni.ru client |
|
||||
| `golang.org/x/sync` | `errgroup` for goroutine lifecycle |
|
||||
| `github.com/teris-io/cli` | CLI framework for `sravnicli` |
|
||||
| `go.opentelemetry.io/otel` (+ exporters) | Tracing & metrics (OTLP gRPC/HTTP, stdout) |
|
||||
| `github.com/stretchr/testify` | Test assertions, suites, and mocks |
|
||||
| `github.com/ydb-platform/ydb-go-sdk/v3` | YDB SDK — **deprecated**, `NewApplication` returns an error for the `ydb` engine |
|
||||
|
||||
## Gotchas
|
||||
|
||||
- `*.json` is gitignored (configs contain local settings). Do not commit
|
||||
config files; document config shape in `cmd/<binary>/config.go` instead.
|
||||
- The `background` binary currently panics on start
|
||||
(`savingOrganizationIDInternalInsteadOfExternal` guard) until the
|
||||
organization-id saving semantics are fixed.
|
||||
- Templ-generated files are not committed; always run `templ generate` (or
|
||||
`task generate`) after pulling or before building/testing.
|
||||
- The YDB engine path is dead code kept for reference — do not extend it; add
|
||||
new storage work to the SQLite adapters.
|
||||
Reference in New Issue
Block a user