English source · 33 min

Testing, coverage and maintenance contracts

External order batching regression

The registered marketplaces suite calls the installed app's real native order callback with 1, 20 and 24 lines. Its app-owned order_localization.py fixture checks frozen names, exact paid amounts, one native outbox event, tenant separation, replay, failed-order rollback and concurrent final-stock admission. Italian, short locale, parent/variant and blank-name cases preserve the previous result. SQLx statement traces assert one translation query at each size and zero on replay. A 25-line request remains rejected before persistence.

The regression was run against the preceding binary first: all behavioral checks passed, and the new query-count assertion correctly rejected [1, 20, 24]. The changed binary passes with [1, 1, 1]. The strict signed_runtime suite repeats the real marketplace callback with separate restricted system and tenant logins, signed request scope and forced RLS. It asserts the same names, stock effects and constant query count. Measurement and delivery scope.

Mail and connected app test ownership

The registered email_tests, rust_connectors and connected_apps suites keep their existing names and execution order. Their shared fixtures and scenarios now have separate owners:

The extraction preserves every mail test body, all 37 connected app assertions and all nine reported scenarios. Before and after the change, the three registered suites pass against disposable local PostgreSQL databases and provider fixtures. The native connector suite retains its two process, encrypted configuration, queue recovery and restricted database role checks. All provider calls stay local.

Run the affected suites through the existing registry:

python3 scripts/verify_integration.py --only email_tests rust_connectors connected_apps

connected_apps retains the archived connector as a differential fixture beside the real Rust commerce process. The separate rust_connectors suite and native mail journey exercise the shipping Rust service. Production ownership remains documented in Rust services; no runtime moved into Python.

Formal and module-boundary regression — 23 October 2026

The current formal/module audit records 63 extracted production policies and 149 Lean properties, including successive stock, refund and budget decisions and saved private agent admission. Compiled Rust/Lean comparison covers 156,741 cases. Four additional Rust tests exercise actual u64 implementations against independent u128 arithmetic and private context truth tables, including u64::MAX and lowered limits.

The complete Linux CI run 39934011164 passed with 317 Rust tests, 816 frontend tests in 140 files, native PostgreSQL/HTTP integration, separate restricted runtime connections and transaction pooling. Measured line coverage is 86.62% Rust, 69.92% frontend and 80.72% Python tooling; all reviewed coverage gates passed. The delayed-flow revocation regression now waits for the committed delay cursor before changing access and the timer, so it checks actual continuation denial instead of racing initial queue admission.

The existing mandatory proof and Cargo checks now also enforce seven handwritten proof domains and reviewed dependency ownership, with negative controls for hidden dependencies, aliases, wrong-owner targets and missing/extra claims. They supplement the existing Rust/frontend size and responsibility checks. Recorded measurements below retain their original dates; consult the linked audit for this revision.

The 17 October buyer/POS follow-up records the complete required 100-suite check and 115 frontend files / 648 tests, plus public preserved-cart, simulated checkout and native POS invoice acceptance. Coverage floors remain unchanged. Public provider payments and a fresh original embedded checkout are not inferred from these local/CI results.

The 17 October native app-field receipt records the complete required Linux check, 99 registered local suites, 636 frontend tests, actual hosted product-app publication and the original embedded simulated checkout. Older measurements below retain their own source/date and coverage scope.

Prototype feedback loop

The public handbook renders Markdown Mermaid fences through the shared site/docs-diagrams.js module. The unmodified Mermaid 11.15.0 browser distribution and its MIT notice are stored in site/vendor/; its package integrity and bundle SHA-256 are recorded in mermaid.json. Build and site checks reject a missing or changed bundle or a disconnected loader. No CDN or network package installation is needed to build or read diagrams, and pages without diagrams do not load Mermaid. Strict rendering disables diagram-defined click actions and sanitizes SVG. Readers can fit wide diagrams to the article, scroll at full size and inspect the original source. Without JavaScript, or when one diagram fails, its source remains readable. Diagram controls follow the handbook's EN/DE/FR/ES language selection.

The 18 October browser repeat rendered all 33 actual handbook diagrams as SVG; two previously invalid sequence labels were repaired without changing their meaning. This real browser check complements the complete static site check: python3 scripts/build_site.py && python3 scripts/check_site.py.

Daily full verification and short per-change checks

Since 29 October 2026, the prototype runs its complete test suite once daily at 02:17 UTC on the default branch. Run workflow starts the same complete suite manually, including on a selected branch when a change needs it. A normal push or PR update no longer starts complete frontend coverage, Rust compilation, Lean/mutation tests, PHP differential tests or PostgreSQL/provider integration.

The required verify name stays in place for branch protection. On pushes/PRs, it means quick checks passed, as stated in the run summary. It does not claim that the current commit passed the full suite. Daily/manual runs must select the full lane; failed, cancelled, missing or unexpectedly skipped jobs still reject admission. Quick runs never download old coverage or label it as current evidence.

PR branches run through pull_request, without duplicate branch-push runs. A new PR revision cancels its old quick run. Daily/manual full runs remain independent. Documentation is built and checked when published from main or requested manually, not again on every PR revision.

Local development follows the same cadence: run targeted tests for changed behavior while fixing it, but do not repeat the whole suite for every commit or wait on nightly tests to continue ordinary prototype work. Run another complete suite only for an explicit acceptance request, a broad regression or a concrete failure needing that scope. Record whether a result is targeted or complete.

scripts/testing/ci_scope.py still classifies actual Git diffs for image admission and retains conservative full selection for scheduled/manual verification. Its rename, missing-history and sensitive-change controls remain in the daily suite.

Image publication scope

The preview and migration image workflows use the same complete Git comparison with ci_scope.py --images before allocating a Docker runner. Only root README/AGENTS/CHANGELOG, handbook/site assets and their rendering tools can skip image work. Package-local Markdown, LICENSE/THIRD_PARTY, frontend, Cargo, fixtures, runtime scripts, Compose, Dockerfiles, workflow changes and unknown paths still build and smoke-test. The registered scope tests check the real Docker COPY inputs against the skipped paths, including rename and multi-commit counterexamples.

Documentation publication and required verification remain independent. A docs-only commit publishes the handbook without creating a new runtime image or moving its tag. Missing history, empty changes and manual workflow_dispatch requests build. If documentation lands while an earlier image is building, that tested image can still advance the preview tag. Newer runtime changes, divergent history and an unavailable comparison prevent promotion. The tag retains the actual built SHA. Use Run workflow to rebuild intentionally; only the main branch can publish. PRs touching image admission exercise the actual images without registry login or publication. Existing packaged signup, private sandbox, session restart and non-root Rust migration health checks still run whenever an image is built.

Complete daily run

After admission, complete frontend verification runs in parallel with the Rust/formal/native jobs. The existing required verify job waits for all full-run results, checks immutable coverage artifact IDs and rejects incomplete evidence. These full jobs are skipped intentionally in the documented quick lane only. No full daily scope or coverage floor was removed by changing the cadence.

Native integration uses three independent CI partitions with separate PostgreSQL, Qdrant, files and ports on separate runners. core runs the ordinary registry; signed runs signed_runtime, including its complete restricted-login HTTP repetitions; pooler runs signed_pooler and transaction_pooler, including their real one-backend PgBouncer checks. The single suite registry remains authoritative. Partition validation rejects duplicates and missing special suites. A newly registered suite automatically enters core; it cannot disappear between jobs.

The app-generation provider fixture additionally owns a fresh disposable database inside its partition. Earlier suites intentionally leave durable tasks for recovery tests; restarting all workers against that shared state could send unrelated requests to a newly rebound loopback model. Generation keeps real workers and the complete provider-wire assertions, starts with no unrelated model requests and drops only its own uniquely named database after stopping its process.

The native collector waits for every partition. It checks source commit, workflow run, producer attempt, Rust toolchain, absolute workspace paths, registry digest and every raw profile checksum before copying any measurement. It rebuilds the matching instrumented objects without rerunning tests. LLVM combines raw Rust profiles and coverage.py combines Python measurements; no percentages are averaged or manually added. The existing final policy and floors remain unchanged. Corrupt, missing or substituted profiles fail the run. A failed matrix job prevents native admission even if the other partitions succeeded.

Local verification remains sequential by default. --partition core, signed or pooler selects a complete partition and rejects --only, --existing-database and --parallel-pooler. The existing optional --parallel-pooler subprocess mode is still available locally with its own disposable Qdrant endpoint. Its owned process group, mandatory join and failure behavior are unchanged.

Run the registered test_integration_partitions, test_partition_coverage and test_parallel_check tooling tests for complete partition admission, rejected coverage substitutions and actual child-process cleanup. Each integration suite continues to print its duration. The successful pre-split native job on 29 October 2026 took 37m13s, of which integration consumed 1781.33s. New wall-clock results must come from a complete successful run, not a projection from isolated suites.

Each job uploads an immutable coverage artifact. The final job downloads the exact IDs returned by those jobs, checks the checkout SHA, workflow run and SHA-256 of each actual report, then runs the existing combined coverage policy and complete source inventory checks. A successful job from an earlier attempt of the same run can be reused when retrying failed jobs. Different commits/runs, future attempts, missing reports and changed contents are rejected. Artifact names include attempts so reruns cannot overwrite earlier evidence. Full HTML reports and integration logs remain available beside the final combined summary; coverage floors are unchanged.

Run python3 scripts/testing/test_ci_gate.py for failed/skipped job and report transfer controls. It is also registered in the native tooling suite. Workflow syntax is checked with actionlint .github/workflows/verify.yml. The 23 October complete run and actual targeted retry passed: frontend feedback after 3m55s, full required verification after 48m15s, and frontend-only repetition with reused native reports after 3m41s. The evidence also records summed runner time and native-test variation; these observations are separate from the older sequential baselines below.

Frontend-only coverage reuses the same policy and source completeness check; it does not fabricate fresh Rust/Python measurements or change their full-run floors. Full runs retain both native separate-login and pooler/multi-replica regressions. Each integration child logs its start and elapsed time, with unbuffered Python output, so a slow suite is visible instead of hidden inside one long CI block. The exact generated source inventory is checked before compilation or integration tests, as well as in the existing structure gate. Missing inventory updates fail immediately without waiting for the daily database suites. The existing reviewed source/build-input registry is also checked before compiling. The later complete formal proof, conformance and mutation checks remain required.

This optimizes development feedback. A green fast check is not a new whole-system acceptance or evidence of live provider payments. The current prototype can use short, coordinated deployment restarts; no zero-downtime fleet migration is required for an ordinary UI or documentation release. Existing demo data is still retained.

Build and rollout time are separate from verification time. The 16 October bustling-bulb-1104 hosted release took 9m12s to build and 11m50s through the complete serialized workflow. Northflank displayed remote cache disabled; its BuildKit disk-cache option was also disabled. No cache capacity was provisioned in this acceptance. The existing Dockerfile separates frontend and Rust stages, so retaining unchanged layers is the next concrete optimization, followed by a measured cold/warm comparison. Northflank's cache guide requires persistent cache capacity and an initial successful population build; enabling a setting alone is not evidence of a faster release.

Measured full-run cost: 21 October

The successful PR #190 full run took approximately 52 minutes. Its timestamped logs show 38.8 minutes in the registered integration step, 4.9 minutes in formal mutation controls, 3.1 minutes in the complete frontend step and 2.1 minutes in the Rust test/build step. Inside the integration step, the signed-runtime repetition took 14.5 minutes and the signed transaction-pooler repetition 3.7 minutes; these are nested timings, not additional time outside the 38.8 minutes.

The succeeding PR #191 full run completed in 45 minutes 29 seconds, including 33 minutes 13 seconds in registered integration tests. All 133 frontend test files / 779 tests passed alongside the Rust workspace, native HTTP/PostgreSQL, formal controls and coverage checks. This run-to-run difference is an observation, not an attributed optimization.

The signed-runtime path intentionally repeats the native HTTP registry using the separate restricted tenant/system logins. The pooler path additionally checks small-pool transactions, current authority and two-replica/key-rotation behavior. Removing these repetitions would remove meaningful production-boundary coverage. The next optimization to measure is concurrent independent database/runner lanes, with separate logs and complete merged coverage. No such speedup is claimed yet; the complete daily/manual run keeps this verification scope unchanged.

Merchant catalogue setup: Current source/upload workflow, native CSV/model/media suites and evidence limits.

Agent authority: Current hosted control, checkout and session acceptance, including separate full/backend and frontend-only source boundaries; older measurements below retain their original dates.

Studio setup and access: Channel wizard fixes, reusable roles, current grants, central B2B configuration and guided company creation.

Fixed shared app storage — 15 October 2026

Storage model, limits and maintenance cutover. The registered app_shared_storage suite creates eight actual shops and installs 32 indexed packages through HTTP. Physical table/index counts remain unchanged. It checks typed keyset filters, equal-ID isolation, six competing unique writes (one success, five HTTP 409 conflicts), protected projections and scoped F5 resets.

app_storage_migration replays the original SQL migrations 001–100 into a separate database and populates historical physical models. Migrations 101–102 preserve inactive apps, personal previews, IDs/revisions, large integers, localized NULL/empty values, cyclic relationships, immutable histories and app-owned document quotas. Missing targets, an unknown dependent SQL view and an idle configured Core login must abort atomically; the successful migration drops only the exact verified old table set without CASCADE and emits no invented business changes.

Existing app, schema, relation, callback, ontology, jobs, assets, native component checkout, staging and visual page/form suites exercise the same store. The strict signed_runtime and real one-backend signed_pooler suites additionally exercise separate non-owner system/tenant logins, signing-key rotation, restart and denied schema/projection access. These are local verification contracts, not proof of a completed production migration or unlimited data/throughput capacity.

The existing identity_broker suite now constructs an actual separate pre-055 database from original SQL and ledger checksums. The native migration process must upgrade it through storage consolidation, install its missing managed Storyfront package and validate the hosted FK before any HTTP read. Two serving processes then expose the original managed connection without changing its package revision or version history. Rewinding only migration 055 on an already post-102 database was an invalid historical fixture; it is not retained as an alternate runtime path.

Agentic directory grids — 14 October 2026

Guide and actual browser captures. The complete local frontend run passes 493 tests in 88 files. V8 coverage measures 64.63% lines, 63.33% statements, 57.23% branches and 52.27% functions across the declared host and trusted app scope. These are measured figures, not 100% coverage. All 243 Rust unit tests, format/all-target Clippy, source/route inventory and four-language gates pass.

Seven affected real PostgreSQL/HTTP/provider suites pass: workspace_roles, merchant_operations, catalog_management, customer_accounts, grid_directory, tenant_isolation and read_performance. Grid counterexamples check whole-directory filters and sorting, localized product-name order, tuple cursors, SQL-looking text, current role/PII denial, foreign tenants/cursors, empty operational staging, invalid model fields and provider failure. HTTP and MCP return the same native rows. Fixtures create customers/orders through native Store API, use isolated databases and a loopback model; no live payment, real customer or paid model call is made.

Chrome acceptance verifies reviewed customer filtering, order detail/return with criteria retained, category/customer-group filters, persisted column ordering, explicit/keyboard popup close, dark appearance and a 390 × 844 viewport without page overflow. App table forms preserve existing actions and use the same layout shell. This does not establish arbitrary app full-text/model support or live-model semantic quality. The existing 54-policy / 156,657-case formal gate passes with zero mismatches and 155 rejected mutations; the grid's SQL, async code and UI are reviewed/tested adapters, not new whole-system proofs. The additional strict-provider counterexample initially failed on unused null fields. Real HTTP/MCP OpenAI Responses fixtures now pass after central configuration and a native restart: only recognized optional nulls disappear, false stays a filter, null identity keys fail closed, and empty supported plans are rejected. These checks use a synthetic loopback key, not an external model account. The rendered site checks 211 pages, all 204 Markdown sources and 5,032 local links/assets, including the illustrated guide and capture provenance.

The full PR run exposed a navigation-test cold-import timing assumption: its default one-second wait expired while the real lazy workspace was still loading. A temporary 1.4-second delay on the actual order-module import reproduces that failure. The corrected readiness wait passes the same delayed path, retains its content/request assertions and bounded budgets, and the diagnostic delay is removed from the committed test. This is functional readiness evidence, not a performance claim. Native workflow/group metadata are now explicit navigation fixtures.

System consistency audit — 14 October 2026

The subsequent legacy-ownership case exposed a separate real admission defect: a new account could see an old email-only guest order. The new regression models the historical persisted shape only in the disposable database. The corrected native account/download owners share one immutable customer-ID SQL predicate; existing digital/PDF tests additionally verify denied claimant access, preserved original guest-cart capability and admitted CRM/merchant access. Unbound old records are not silently assigned or financially rewritten. SQL execution remains outside the selected Lean proofs.

The identity-class follow-up also reproduces a forged verified-purchase badge through typed guest email. The corrected review producer and native numberOfReviews rule consumer are tested together: guest-email inheritance is excluded, actual account submission is counted, and the real original cart can still review its purchase. All nine affected native suites pass. A separate native migration-only upgrade confirms 093 withdraws the old badge while retaining approval/text/rating.

Second pass and full component matrix: 49 registered native HTTP/PostgreSQL/provider suites pass at b4fb561, followed by a reproduced history-discovery counterexample and its targeted repair. MCP now requires the same entity read/write/PII conjunctions as invocation. Rust format, all-target Clippy and 243 tests pass. Private Experience 6563b2d passes its complete check with 230 tests, type/structure/localization checks and build. The private entry-chunk size warning remains visible. This refresh adds no paid provider calls, fresh dense-Qdrant test, public full-flow acceptance or claim of 100% coverage. The targeted repaired roles/history/isolation rerun passes, including the new denied-access cases. The regenerated site validates 211 pages, all 204 Markdown sources and 5,036 local links/assets. The previous pass's evidence remains dated below.

The old empty-registration public probe failed before native policy admission. hosting_check.py now uses the native DTO shape with deliberately invalid email and an empty password, so no account/shop can be created even if signup is open. The refreshed public health/unauthorized-operator/closed-signup checks pass. Fifteen tooling tests cover expected JSON, non-JSON denials, wrong status/shape and oversized responses; failures retain bounded diagnostics rather than becoming successful gateway claims. Typed-body extraction can still produce a plain-text 422. This is not completed public onboarding or authenticated end-to-end acceptance.

Full findings and boundaries. Thirteen selected isolated HTTP/PostgreSQL/provider suites passed before new counterexamples exposed knowledge-source stage rights and HTTP/MCP automation admission defects. The repaired paths passed seven affected suites plus new narrow-role regressions in testing/consistency_boundaries.py. Existing tests alone had missed those cross-boundary cases. All 242 Rust tests, format/all-target Clippy, exact source/route inventory and existing 54-policy / 156,657-case formal gates pass; the mutation gate rejects 155 broken policies. Private Experience's complete check passes 230 tests. No new coverage percentage or whole-system guarantee is inferred; no live provider or paid AI call was made. The rebuilt documentation passes 208 pages, all 201 Markdown sources and 4,957 local links/assets, including this audit.

Fulfillment, carrier app and identity verification — 14 October 2026

The local fulfillment run passes 483 frontend tests in 87 files and 242 Rust unit tests across the native library/core and independent service binaries. Measured frontend coverage is 64.12% lines, 62.84% statements, 56.89% branches and 51.56% functions. These figures are dated local measurements, not 100% coverage.

Real PostgreSQL/HTTP/MCP suites cover package quantity/price/stock conservation, concurrent replay, current rights, customer projection, one Flow effect, selected package delivery-note PDFs, immutable private carrier documents, quota recount and atomic saturation denial. A carrier-neutral test-only service drives actual native jobs, PDF/tracking, confirmed-no-effect release and unknown-effect allocation/dispatch fences across a native restart. A real split event drives a current-actor idempotent Flow job and attaches one original label. A shipping clerk cannot read another actor's job and loses private results immediately on permission removal. No live carrier, email or paid booking is used. Source ownership, route rights, inventory and localization checks pass; shipping setup and limitations. The rendered documentation checks 207 pages, all 200 Markdown sources and 4,919 local links/assets. Manual Chrome acceptance confirms one click produces two native packages with conserved order value and a package-specific delivery-note selector.

Native merchant/operator MFA, verified-email recovery and scoped handoffs were landed separately; their source/HTTP/browser checks are recorded in account security. The combined local formal check covers 54 extracted policies and 156,657 compiled Rust/Lean cases with no mismatches; SQL, provider effects and the rest of the runtime remain unproved.

Studio setup and access verification — 13 October 2026

The isolated local run passes 477 frontend tests in 85 files, plus 232 Rust unit tests across the library, core and bundled service binaries. The frontend V8 report includes the declared host and trusted app source scope: 64.04% lines, 62.73% statements, 56.93% branches and 51.34% functions. These are measured local values, not 100% coverage or confirmation of public deployment.

Real PostgreSQL/HTTP checks cover channel creation, role changes and concurrent delegation, current key/callback grants, PII denial, tenant isolation, native B2B/UCP setup and feature switches. Browser checks use synthetic accounts and confirm POS/marketplace creation, preserved App Studio drafts, company/customer linking, B2B settings and read-only product navigation. The complete documentation build checks 203 pages, all 196 Markdown sources and 4,983 local links/assets.

Connected feature audit — 12 October 2026

Integration map and repaired boundaries. Real isolated PostgreSQL/HTTP regression runs cover POS, B2B history and policy staging, inherited analytics settings/capture, selective release, settings scopes, customer history, foreign-tenant references, legal consent, currency behavior, native/app automation, EDI and marketplace provider fixtures. Denied writes inspect unchanged live state; B2B staging checks current live spending and preserves employee grants. App Flow publication checks the current installed target, action-specific rights and activation. No real AI, customer email, payment or seller account is used by these fixtures.

The current frontend run passes 470 tests in 84 files. The complete Rust unit run passes 224 tests across the library, Core and bundled service binaries; formatting, all-target lint and production builds pass. The existing 51 extracted policies match compiled Rust/Lean in 155,347 cases, with no mismatch; this adds no whole-server proof. The documentation build checks 200 pages, all 193 Markdown sources and 4,850 local links/assets. Explicit API rights, modular source inventory and four-language gates remain required. Dated coverage numbers below belong to their original source/CI; this audit does not claim 100% coverage or whole-system proof.

MCP stack regression — 13 October 2026

The complete Linux coverage build found a worker stack overflow at MCP pos.cart after native POS setup succeeded. The server log confirmed stack overflow rather than a payment/order failure. Small future-state size checks failed to expose the much larger unoptimized poll frame, so they are not an acceptance test.

The shared dispatcher now chooses a boxed domain future synchronously before its child is polled. Existing real pos and b2b HTTP/PostgreSQL/MCP suites pass locally with RUST_MIN_STACK=1572864 (1.5 MiB), including sales, receipts, history and policy publication. This is a constrained local regression run, not a portable stack-size guarantee. Production stack limits remain unchanged; full Linux coverage CI must pass before release.

Behavioral intelligence verification — 12 October 2026

The new registered behavior_intelligence suite runs real PostgreSQL/API/MCP requests with synthetic guests and customers, native cart changes and a simulated order. It rejects forged identity/purchases, foreign tenants, conflicting replay and consentless intake; checks independent withdrawal, self-export/erase, expired reads, hourly admission and ledger RLS with NOSUPERUSER NOBYPASSRLS. The existing tenant_isolation suite additionally checks all four new composite foreign keys. Related legal/account/knowledge suites remain green. Six UI regressions cover consent, scope reset, links, graph keyboard access, failed reload, search privacy and stale private-journey responses. Desktop/mobile screenshots use an isolated synthetic shop, with no paid models, customer mail or real payment calls. Ownership and functional boundaries.

The prototype has automated behavior checks, actual PostgreSQL/Qdrant integration (plus legacy conversion checks), original Shopware comparisons and partial Lean contracts. It is not fully tested at 100%, and neither coverage nor the Lean subset proves the entire system bug-free.

Current acceptance (9 October 2026)

Complete change and end-to-end record joins the source owners, all registered suites, actual browser/private purchase checks and remaining audit work. The completed merged-Core CI at 598bbec measures Rust 90.55% lines / 87.94% functions / 87.24% regions, frontend 63.23% lines / 54.78% branches / 51.14% functions / 61.94% statements and Python tooling/reference scope 81.34% lines / 57.80% branches. Exact source/CI artifact summary.

Dated measurements below retain their original source. That dated suite had 187 Rust unit tests and 388 frontend tests; the private Experience application and original Storyfront container have separate evidence and no claimed percentage.

Source architecture

• Rust: 414 source modules (merged cognitive release, 9 October 2026), each with a responsibility header, at most 320 lines; main.rs at most 120. Existing domain folders remain independent of extension app implementations.
• Frontend: separate admin/, storefront/, platform/ and shared/ ownership. Views/controllers and styles are limited to 400 lines; locale data has a documented 700-line allowance. Runtime cycles, unresolved local imports, crossing application boundaries, missing folder contracts and undocumented source files fail CI.
• Bundled Email/GA4/Gmail/Slack implementations are Rust modules under their extensions/apps/<app>/ package; shared service hosting is in extensions/services/connectors/src/; independent app examples and browser SDKs remain under extensions/. Python test tooling lives under scripts/ and the previous connector is an archived comparison oracle in reference/. The generated inventory covers runtime/tooling sources and is checked for drift.
• Studio workspaces load lazily. Root application routing is isolated in frontend/src/application/; error boundaries keep workspace failures inside the current view and provide reload recovery for rejected cached module imports. The removed CommerceManager had no call site and duplicated old operational UI. Order state management remains in admin/orders/OrderWorkflow.tsx; product review moderation lives in admin/catalog/ReviewModeration.tsx.

frontend/tests/architecture.mjs also inserts deliberately broken synthetic imports, a cycle, an oversized file and absent documentation into temporary directories; it verifies that the architecture checker actually rejects each defect.

Repeatable checks

From the repository root:

cargo fmt --check
cargo clippy --locked --all-targets -- -D warnings
cargo test --locked
cargo build --locked --bins
npm --prefix frontend ci
npm --prefix frontend run format:check
npm --prefix frontend run build
npm --prefix frontend run architecture
npm --prefix frontend run localization
npm --prefix frontend run test:coverage
python3 scripts/formal.py
python3 scripts/formal/mutations.py
QDRANT_URL=http://127.0.0.1:16333 python3 scripts/verify_integration.py

Local integration creates and removes its own uniquely named synthetic database; existing shops remain outside that run. The default database container is veyndo-postgres-1; override it with --container. It uses DATABASE_URL from the process or the local .env. CI uses the same owned disposable database path. --existing-database remains an explicit local disposable-fixture override. Failures remain failures, and child processes stop before database cleanup. SIGINT flushes optional Rust coverage profiles.

scripts/testing/suites.json is the single registry for HTTP suites, local provider and connector suites, browser contracts and verification tools. It records the current suite names and execution order. The local server uses an offline model URL; live model checks are separate, opt-in checks. Credentials, payments, mail and Slack are exercised against loopback protocol fixtures. Passing these does not demonstrate a real provider account.

The currencies suite exercises channel restrictions, Store API/MCP/UCP agreement, fixed and converted prices, shipping/coupon conversion, stale checkout reviews, immutable orders and grouped revenue. payment_providers tests create/capture/refund with EUR/USD, zero-decimal JPY and three-decimal KWD against a local protocol fixture. production_foundations denies cross-tenant currency-job SQL access using non-owner roles, then restarts a worker after its first 100-product batch and verifies completion, exact midpoint rounding and exclusion of products created after the job snapshot. See multi-currency commerce for the configuration and proof boundaries.

The original-PHP comparisons and reflected automation catalog checks are:

python3 scripts/differential.py
python3 scripts/context_differential.py
python3 scripts/delivery_differential.py
python3 scripts/rule_differential.py
python3 scripts/automation_registry.py
python3 scripts/automation_differential.py
python3 scripts/money_boundary_differential.py

The automation suite also executes scripts/playground.py as documented: a new owner-owned shop, repeat setup after an edited rule, both checkout branches and actual invoice records. tooling_tests.py checks refusal of remote credential destinations and unsafe state file symlinks, alongside runtime ownership and coverage reporting contracts. See the manual tour for the corresponding browser steps.

Production boundary regressions

production_foundations runs two non-owner, non-bypass PostgreSQL replicas and repeats native checkout, CRM and adversarial tenant checks. It rejects unsafe TRUNCATE grants, omitted/forged tenant predicates, stale operator quotas and corrupted stock snapshots; it tests commit-only cache eviction and daily AI usage shared across live/staging. The 32-client scalability fixture explicitly configures 32 tenant permits. Production defaults and saturation policies remain configurable. The architecture guide explains exactly what these checks and the Lean subset establish.

Component regressions

frontend/tests/unit/ uses real components/hooks/transports, synthetic domain fixtures, jsdom and Testing Library. Fetches fail by default unless explicitly supplied by a test. Cases include:

• merchant authentication, chat failures, language changes, uploads and separate live/staging transports;
• navigation through permission-filtered lazy Studio workspaces and a signed-out login path;
• catalogue filtering, cursor pages, stale responses, customer-session expiry and forbidden operation replay;
• independent default addresses, linked customer/order/product back paths, configured translated groups, lazy history and confirmed revision-bound restoration with retained failed drafts;
• opt-in personalization, ranking, consent withdrawal and failed signals;
• product gallery, variants, normalized purchase quantities, tier prices, SEO, reviews and public source-backed product questions;
• delayed product answers after navigation and product-specific moderation with rejected writes;
• connected flow branches/deletion, frozen execution traces, incomplete JSON drafts and source condition round-trips in four languages;
• SMTP/Resend/SendGrid fields, saved secrets, read-only settings, translated templates and explicit dry-run test calls.

Coverage collection

Frontend coverage includes every frontend/src/**/*.ts and frontend/src/**/*.tsx, plus TypeScript/React modules owned by extensions/apps/**, including modules never imported by a test. HTML, JSON, LCOV and summaries go to frontend/coverage/. App source globs resolve to absolute package paths and V8 external-file collection is enabled; the report rejects omitted app modules as well as omitted host modules. Styles and independent guest app bundles are outside that statement metric and explicitly listed as separate scopes.

For Rust install cargo-llvm-cov 0.9.1 and llvm-tools-preview. Use the same instrumented binaries for units, original-PHP comparisons and the actual HTTP integration run:

export CARGO_LLVM_COV_TARGET_DIR="$PWD/target"
cargo llvm-cov clean --workspace
eval "$(cargo llvm-cov show-env --sh)"
cargo test --locked
cargo build --locked --bins
# Run the integration and comparison commands above, then:
mkdir -p artifacts/coverage
cargo llvm-cov report --json --output-path artifacts/coverage/rust.json
cargo llvm-cov report --html --output-dir artifacts/coverage/rust

Mutation variants write profiles outside the real-core report directory; deliberately broken code cannot inflate the production coverage metric.

The Rust report measures lines, functions and regions, not branch coverage. The JSON preserves every module in this instrumented scope. Do not call covered regions exhaustive branch coverage.

Python instrumentation is a development dependency, not a production service dependency. It collects subprocesses and includes untouched namespace modules:

python3 -m pip install -r scripts/testing/requirements.txt
COVERAGE_PROCESS_START="$PWD/scripts/testing/.coveragerc" PYTHONPATH="$PWD/scripts/testing" python3 scripts/verify_integration.py
python3 -m coverage combine --rcfile=scripts/testing/.coveragerc
python3 -m coverage json --rcfile=scripts/testing/.coveragerc
python3 -m coverage html --rcfile=scripts/testing/.coveragerc
python3 scripts/testing/coverage_report.py

The report covers Python app/service sources and verification tools, so its percentage must not be presented as coverage of the Python services alone. Before a fresh standalone Python measurement, remove only the prior synthetic artifacts/.coverage* data, to avoid mixing historical executions.

CI uploads complete reports and lists zero-hit modules and unmeasured scopes. coverage-policy.json locks the measured regression floors; Studio transport, server-health and workspace-boundary modules require 100% lines, branches, functions and statements. Adding a source file counts in the denominator. Raising those floors is an explicit reviewed change; do not silently lower them to obtain a green check.

Explicit 100% audit and open work

npm --prefix frontend run test:coverage:full
python3 scripts/testing/coverage_report.py --require-full

These commands currently fail, as they should. The second also rejects a full system claim while browser SDK/example-app JavaScript and Wasm instruction coverage remain unmeasured. CSS requires visual behavior checks; live OAuth credentials, external inference and production deployment need their own verification.

The initial measured snapshot is in quality-baseline.json. The largest remaining frontend gaps include order-detail edits/documents, customer/address edge cases, full operator workflows, app iframe errors and complex rule/flow interactions. Coverage HTML shows individual missed statements and branches, not only totals. Future changes must add behavior regressions in these owning modules and increase coverage; a passing threshold is not completion of the 100% objective.

Lean source review locks and mutation checks continue in CI unchanged in scope; see formal verification for precisely what is proved.

The account composition regression exercises both login and registration through CustomerAccount → CustomerSignIn → shopApi. The simulated transport rejects protected profile/address/order reads unless the form persisted the exact tenant session key. This reproduces the former React-reserved key prop bug; backend address/customer isolation is separately exercised over real HTTP.

Studio session regressions reproduce a product 401 after a previously loaded overview, verify that all private controller state is cleared, and reconnect with a renewed token. Shared merchant calls and focus revalidation exercise the same path. Negative cases cover 403/5xx, customer/provider/operator errors, invalid login attempts, old-token replies after reauthentication, coalesced reads, hidden tabs, overlapping checks and timer/listener cleanup. Browser verification uses the existing personal test account; no expiry bypass or new privileges are added.

International configuration regression scope (2026-10-05)

New PostgreSQL suites cover complete bundled geography, custom countries/regions, US destination taxes in actual product/cart/order paths, saved-rule conditions, non-English main-language inheritance and stale/foreign configuration rejection. Translation fixtures cover Ollama/OpenAI/Claude wire formats, durable full-catalogue drafts, cold restart, stale apply, repeated apply and permission/tenant denial. They perform no paid model calls. React tests exercise keyboard/group country selection, missing-field inheritance and a shared settings draft across navigation. UI screenshots supplement these tests; neither replaces whole-source coverage.

Recorded international-commerce verification

CI run 37285967720 verified the international-commerce tree merged as 852f3d1 on 5 October 2026: 67 Rust unit tests, 90 frontend tests and the registered 25 HTTP / 3 provider / 4 browser-contract suites. The complete measured report, including untouched modules and unmeasured scopes, is checked into quality-baseline.json.

These percentages retain the separate source scopes above and do not establish 100% coverage or whole-system correctness. Regression floors were preserved.

Company profile HTTP/database checks run in the suite registry as company_settings; the scope, decoder/ACL, issuer snapshot, translation and selective release cases are detailed in company settings. They use synthetic tenants and local image bytes, with no external payment/model/OAuth traffic.

Channel settings and media workspaces (2026-10-05)

See the settings/media guide for the current single-language editor, field-level checkout overrides, dependency-safe method removal, gallery and optional private image jobs. These are native prototype extensions; they do not establish additional full Shopware API/DAL parity, current tax law, paid-provider quality or whole-system formal certification.

App library regression scope (2026-10-05)

Eight component regressions exercise translated search, categories/status/discovery, installation followed by the real detail page, cancellation and failed activation, viewer restrictions, registered native interfaces, locale inheritance and independent artwork failure fallbacks. The isolated PostgreSQL app suite checks metadata round trips, tenant separation, immutable same-version digests, unsafe artwork refusal and the actual shop main language. Strict model-schema regression keeps optional presentation compatible with both provider request formats.

The app-library clean frontend run passed 137 tests: 50.16% statements, 43.76% branches, 41.35% functions and 50.98% lines across all included source. These measurements are not whole-system 100% coverage. Actual browser checks supplement component tests with desktop and 375px layouts. The affected apps, app surfaces, App Studio and developer-document HTTP suites pass using local fixtures, without paid calls.

Connected knowledge workspace (2026-10-05)

The current clean frontend run passes 151 tests across 26 test files, including ten knowledge regressions: whole-shop census, a single inherited-language editor, explicit empty values, non-English main language, guarded publication, viewers, retrieval scope, failed edits, actual product navigation and recommendation review. Across all included source the run measures 52.16% statements, 45.88% branches, 43.10% functions and 52.99% lines. This is a measured frontend scope, not a whole-system coverage update or a 100% claim.

84 Rust unit tests, strict lint/format, the extraction/Lean gates and negative mutations pass. The affected isolated PostgreSQL/AGE suites (knowledge_workspace, developer_documents, apps, staging, connected_apps, automation) exercise source hashes, public/private/archive fences, enabled-language retrieval, >50-source pagination, optimistic revisions, graph reassignment, granular HTTP/MCP access and selective release. Local provider fixtures prove private document passages reach the actual merchant model prompt and disappear after archive. No paid provider calls are used.

The knowledge suite also follows source ingestion through a product-scoped event rule into a durable completed flow without an order. Four UI regressions retain the stable trigger IDs while translating their labels EN/DE/FR/ES.

Browser checks cover inherited source creation in App Studio Lab, private customer exclusion versus merchant retrieval, the real product preview, desktop and 375px layouts. See the knowledge guide for source ownership, HTTP/MCP contracts, screenshots and the fixed-model/causality boundaries.

Adversarial tenant isolation

The registered tenant_isolation HTTP suite uses two synthetic workspaces, real merchant/customer sessions, direct MCP/UCP calls, foreign IDs, forged scope headers, same-ID app records and revoked one-workspace integration keys. It re-reads victim state after denied mutations. The database helper tests all 22 new composite foreign keys with own/foreign controls, scans tenant foreign keys for missing scoped counterparts, and tests forced app RLS under a restricted role including context reset on the same connection. It runs automatically via the existing CI integration registry, with zero provider calls.

Exact scope and remaining core-RLS boundary. Grouped checks are behavioral evidence, not 100% endpoint/authorization coverage.

Checkout verification (6 October 2026)

The checkout change passes 226 frontend tests across 34 files (including twelve checkout regressions) and 96 Rust unit tests. The preceding 75bd24a whole-source frontend coverage snapshot measured lines 60.78%, statements 59.70%, branches 53.50% and functions 49.32%. These figures are frontend-only and do not replace the older full-stack baseline report.

Real isolated SQL/HTTP suites verify review admission (five checks), native payment wire behavior (ten checks) and tenant isolation (38 checks). The browser saved a synthetic order after explicit review; 390-pixel emulation had no checkout content overflow. No paid model/PSP calls were used. The public Studio's origin fix was verified separately with allowed-origin 200, foreign-origin 403 and a fresh merchant-owned test shop with products, eight discoverable apps and browser MCP.

Platform control-plane regressions (6 October 2026)

platform.py runs platform_control.py against disposable PostgreSQL shops and a local synthetic model service. Sixteen combined checks include independent operator grants, two-shop/fresh-process encrypted provider inheritance, stale revisions, pause/background deferral, reversible trash, Host-based routing and persisted measured-call latency. No external model or payment call is made.

shop-scope.test.ts covers public subdomain upgrades and protected/local URL exceptions. platform-control.test.tsx covers independent entry links, write-only key editing and typed-ID/reason confirmation before trash. The complete frontend unit run passed 38 files / 261 tests; Rust passed 100 tests. Affected provider, transaction/payment, app/flow/staging, translation/image/document/search and tenant-isolation integration suites also run through their real local paths. These checks are not live provider capability or a full-system security proof.

managed_search.py allocates its own loopback port, so an open local Studio fixture cannot accidentally satisfy its readiness check. Qdrant-backed suites require the actual configured local Qdrant address.

Core hardening regressions (8 October 2026)

The existing strict-runtime suite now checks persistent login backoff across two processes, poison-event savepoint isolation and recovery, referenced/unsettled retention, and database-wide admission saturation/expiry. transaction_pooler repeats real checkout, customer registration/login, CRM, app/staging and RLS paths through one actual PgBouncer backend; listeners stay on a direct runtime connection. This exposed and removed nested pooled borrows under checkout/login locks. CI installs PgBouncer 1.21+ and uses the same suite registry. Native execution may select TEST_PSQL and TEST_PGBOUNCER; all databases and pooler processes are isolated fixtures and cleaned up. Unknown route rights and trusted Principal extraction also have unit/static controls. Implementation and boundaries.

These checks establish those behaviors, not 100% whole-system coverage, public rollout status, a speed-up percentage or protection against a compromised server.

App-platform v2 verification

Registered PostgreSQL suites exercise app_consent, app_callbacks, app_preview, app_relations, app_schema, app_jobs, app_components, app_assets, app_secrets, app_events, app_export and app_distribution. They observe installed versions and actual records/DDL, denied foreign mutations, lease fences, provider signatures, replay and export bytes. Existing apps, services, staging, connectors and automation suites verify compatibility.

Frontend tests exercise native code-behind, action allowlists, actor draft restore, revision conflicts, iframe bundle rejection, controls and package consent. These are regression evidence, not a claim of 100% app/host coverage. Full semantic document tests additionally require the real Qdrant service; a missing retrieval service must be reported as an unverified environment dependency, never replaced by a mocked successful semantic result. See the security matrix.

App ownership checks

Architecture, localization, translation export and frontend coverage include trusted app UI under extensions/apps/. The Rust source inventory and verification review ledger include app and service Rust modules. scripts/testing/plugin_ownership.py checks canonical package layout and has rejecting controls for leaked core files and escaping symlinks. Provider and checkout integration suites continue to test existing HTTP/MCP/Flow behavior after relocations. This is source ownership and regression coverage, not a proof that every trusted compatibility adapter executes in a sandbox. See exact boundaries.

EDI export identity — repeated consistency audit, 14 October 2026

A full CI run exposed an intermittent repeated-export sourceHash mismatch. The deterministic existing EDI suite now adds a native operational order note between exports. Before repair, only the source hash changes while the document is identical. After repair, the app fingerprints the actual serialized document and explicit native source IDs/format/kind; the complete response stays identical. Real EDI and native B2B suites, seven EDI Rust tests and targeted Clippy pass. The original failure is retained; no equality assertion was weakened.

A separate read-only hosted check of Core 540e066 found the mandatory RLS switch unset and no separate runtime credential. This is an open hosting configuration issue, not evidence that local strict-RLS tests enable production protection. See the complete status and prepared provisioning boundary.