kurious/AGENTS.md

AGENTS.md

Guidance for AI agents (and humans) working in this repository.

Project Overview

kurious (git.loyso.art/frx/kurious) is a course/education platform that aggregates and serves educational course listings. It scrapes/syncs course data from an external source (sravni.ru) via a rate-limited HTTP client, stores it locally, and exposes it through a server-rendered web UI with filtering, pagination, and statistics.

The project is written in Go and follows a hexagonal (ports & adapters) architecture with a CQRS-flavored application layer.

Tech Stack

• Language: Go 1.26 (toolchain go1.26.4; see go.mod)
• HTTP routing: github.com/gorilla/mux
• HTTP client: github.com/go-resty/resty/v2 (sravni.ru scraper)
• Database: SQLite via modernc.org/sqlite (pure-Go, CGO disabled). YDB was historically supported but is no longer supported (service.NewApplication returns an error for the YDB engine).
• DB access: github.com/jmoiron/sqlx (named queries)
• Templating: github.com/a-h/templ (.templ files compile to Go)
• Observability: OpenTelemetry (go.opentelemetry.io/otel) — traces and metrics with stdout / OTLP (HTTP & gRPC) exporters
• Background jobs: github.com/robfig/cron/v3
• Logging: standard log/slog (text or JSON, configurable)
• Rate limiting: golang.org/x/time/rate
• Testing: github.com/stretchr/testify (assert, require, mock)
• Mocking: github.com/vektra/mockery/v2 (config in .mockery.yaml)
• Linting: golangci-lint v1.55.2
• Build/task runner: Taskfile (Taskfile.yml)
• Build flags: ldflags inject version, commit, buildTime (see kurious.go)

Architecture

The codebase implements a hexagonal architecture (ports & adapters) with a CQRS-style separation between commands (writes) and queries (reads).

                 ┌─────────────────────────────────────────────┐
   delivery      │  ports/        (HTTP server, cron jobs)     │
   mechanisms    └──────────────────────┬──────────────────────┘
                                         │ depends on
                 ┌──────────────────────▼──────────────────────┐
   application   │  app/          (command/, query/)           │
   layer         │  app.Application { Commands, Queries }      │
                 │  decorator/    (logging decorators)         │
                 └──────────────────────┬──────────────────────┘
                                         │ depends on
                 ┌──────────────────────▼──────────────────────┐
   domain        │  domain/       (entities, repository ports) │
                 └──────────────────────┬──────────────────────┘
                                         │ implemented by
                 ┌──────────────────────▼──────────────────────┐
   adapters      │  adapters/     (sqlite_*, memory_mapper,    │
                 │                  ydb_* legacy stub)         │
                 └─────────────────────────────────────────────┘

• domain/ — pure business entities (Course, Organization, LearningCategory) and the repository interfaces (ports) that the application depends on: CourseRepository, OrganizationRepository, LearningCategoryRepository. No I/O or framework code lives here.
• app/ — the application layer. app.Application aggregates a Commands struct and a Queries struct. Handlers in app/command/ perform writes; handlers in app/query/ perform reads. Each handler implements a generic decorator.CommandHandler[T] or decorator.QueryHandler[Q, U] interface and is wrapped with logging decorators at construction time.
• adapters/ — concrete implementations of the domain repository ports. sqlite_course_repository.go, sqlite_organization_repository.go, etc. translate between domain types and SQL rows. memory_mapper.go is an in-memory mapper translating external dictionary IDs to human-readable names.
• ports/ — delivery mechanisms. ports/http/ is the HTTP server (gorilla/mux
  • templ templates); ports/background/ runs scheduled cron jobs (e.g. SyncSravniHandler).
• service/ — the composition root. service.NewApplication selects the DB engine, constructs the SQLite connection, builds the adapters, and wires everything into an app.Application. This is what binaries call.

Data flows: an HTTP request enters ports/http, calls a handler on service.Application (which delegates to app/command or app/query), which calls a domain repository interface implemented by an adapters/* repository.

Directory Structure

.
├── kurious.go                 # Root package: version/commit/buildTime getters
├── Taskfile.yml               # Build, test, lint, generate, run task definitions
├── .mockery.yaml              # mockery config (with-expecter, keeptree)
├── go.mod / go.sum
├── cmd/                       # Entry points (one package per binary)
│   ├── kuriweb/               # Main HTTP web server (config.go, main.go, http.go, trace.go)
│   ├── background/            # Background sync process (cron-driven sravni sync)
│   └── dev/sravnicli/         # Developer CLI for inspecting the sravni source
├── internal/
│   ├── kurious/               # Application core (the hexagon)
│   │   ├── domain/            # Entities + repository interface ports
│   │   ├── app/
│   │   │   ├── app.go         # Application{Commands, Queries} aggregate
│   │   │   ├── command/       # Write-side handlers (Create, Delete, Update...)
│   │   │   └── query/         # Read-side handlers (List, Get, Stats...)
│   │   ├── adapters/          # Repository implementations + mocks/
│   │   ├── ports/
│   │   │   ├── http/          # HTTP server, course routes
│   │   │   │   └── bootstrap/ # templ templates (.templ) + generated _templ.go
│   │   │   ├── background/    # Cron job handlers
│   │   │   ├── background.go  # BackgroundProcess (cron scheduler)
│   │   │   └── services.go    # Services{HTTP, Background} aggregate
│   │   └── service/           # Composition root: NewApplication wiring
│   └── common/                # Shared, reusable utilities
│       ├── client/sravni/     # sravni.ru HTTP client + mocks/
│       ├── config/            # Config structs (HTTP, Log, Sqlite, Trace, YDB, Duration)
│       ├── decorator/         # Command/Query handler interfaces + logging decorators
│       ├── errors/            # SimpleError, ValidationError, sentinel errors
│       ├── generator/         # ID generators
│       ├── nullable/          # Generic nullable Value[T]
│       ├── xcontext/          # context helpers (request id, log fields)
│       ├── xlog/              # slog + cron logger adapters
│       └── xslices/           # slice helpers (Map, Filter, ForEach, LRU)
├── pkg/
│   └── xdefault/              # WithFallback helper (public-ish pkg)
├── migrations/
│   └── sqlite/                # SQL migrations (embed.FS) + migrator.go
├── assets/kurious/static/     # Static web assets (embedded via go:embed)
└── htmlexamples/              # Standalone HTML/templ prototyping examples

Build, Test, and Lint Commands

All operations are driven by Taskfile (task <name>). The toolchain is installed into a local bin/ (GOBIN={{.USER_WORKING_DIR}}/bin) and CGO_ENABLED=0 is enforced.

Command	What it does
task install_tools	Install golangci-lint, templ, mockery into bin/
task generate	Run templ generate (compiles .templ → _templ.go)
task mocks	Run go generate ./internal/... (regenerate mockery mocks)
task check	Run golangci-lint run ./... (depends on generate)
task test	Run go test ./internal/... (depends on generate)
task build_web	Build bin/kuriousweb (depends on check + test)
task build_background	Build bin/kuriousbg (depends on check + test)
task build_dev_cli	Build bin/sravnicli (depends on check + test)
task build	Build all three binaries
task run	Build then run bin/kuriousweb

Typical workflow before committing: task check && task test.

Running the binaries

Binaries read a JSON config file path as their first argument, defaulting to config.json (note: *.json is gitignored, so configs are local). Example fields: log, sqlite, http.listen_addr, db_engine, tracing.

./bin/kuriousweb path/to/config.json

Code Conventions

Decorators / handler pattern

Command and query handlers implement generic interfaces and are wrapped with logging decorators at construction time. When adding a new use case:

1. Define the command/query struct and a *Handler type alias of decorator.CommandHandler[T] / decorator.QueryHandler[Q, U] in app/command/ or app/query/.
2. Implement a private *Handler struct holding its repository dependencies.
3. Expose a New*Handler(deps..., log *slog.Logger) constructor that returns the decorated handler (decorator.ApplyCommandDecorators / decorator.AddQueryDecorators).
4. Register the new handler on app.Commands / app.Queries in app/app.go and wire it in service/service.go.

Mock generation (mockery)

• Interfaces marked with //go:generate mockery ... directives (see domain/repository.go, common/client/sravni/client.go) are mocked into sibling mocks/ packages (keeptree: True, with-expecter: true).
• Regenerate with task mocks (runs go generate ./internal/...).
• Mocks are committed to the repo.

Templating (templ)

• HTML is authored as .templ files under internal/kurious/ports/http/bootstrap/.
• task generate compiles them to *_templ.go (committed). Run it whenever a .templ file changes; task check and task test both depend on it.

Database / migrations

• SQLite migrations live in migrations/sqlite/*.sql and are embedded via go:embed. The migrator (migrations/sqlite/migrator.go) applies them in order inside a transaction.
• Repository adapters use sqlx named queries; domain ↔ row translation is centralized via AsDomain() methods.

Logging & observability

• Use log/slog with structured attributes. Request-scoped fields (e.g. request_id) are propagated through context via internal/common/xcontext.
• HTTP handlers wrap requests in trace spans and record metrics (request duration histogram). Database adapters emit spans tagged with db.* attributes.

Error handling

• Domain/repository errors use sentinels from internal/common/errors (ErrNotFound, ErrNotImplemented) and *ValidationError (mapped to HTTP 400/404 in ports/http/server.go).
• Wrap errors with fmt.Errorf("doing X: %w", err) to add context while preserving the underlying error for errors.Is / errors.As.

Testing Patterns

• Scope: unit tests live next to the code they test (*_test.go). task test runs go test ./internal/....
• Framework: github.com/stretchr/testify — require for fatal assertions, assert for non-fatal, mock for mock interactions.
• Mocks: generated mockery mocks with the expecter API:

  repo.EXPECT().Create(mock.Anything, expectedParams).
      Return(domain.Course{ID: "c1"}, nil).Once()
  

  Note: when the generated mock does not yet implement every interface method, tests embed the mock and add the missing methods (see the fullRepo pattern in internal/kurious/app/command/command_test.go).
• Logger: tests use a quietLogger() helper that discards output (slog.New(slog.NewTextHandler(io.Discard, nil))).
• Adapter tests: internal/kurious/adapters/sqlite_*_test.go exercise the real SQLite repositories against an in-memory/temp database.
• Conventions: table-driven where appropriate; use context.Background() in tests; assert on wrapped errors with assert.ErrorIs and message fragments with assert.Contains.

Notes for Agents

• Run task generate before linting/testing if you touched any .templ file.
• Do not re-introduce YDB as a working engine without updating service.NewApplication and the migration story (currently SQLite-only).
• Generated files (*_templ.go, mocks/*.go) are committed — regenerate and commit them alongside source changes.
• *.json, *.sqlite, bin/, and *.log are gitignored; do not commit local configs or databases.