Changelog
1.1.1 — 2026-10-08
Section titled “1.1.1 — 2026-10-08”Threadmill 1.1.1 is a patch release for the Oracle store. See Upgrade from 1.1.0.
- Fixed the Oracle baseline migration failing with ORA-12899 at statement 28 of
V1__baseline.sqlon databases withMAX_STRING_SIZE=EXTENDED. Thethreadmill_concurrency_groups.idle_sortvirtual column declaredRAW(2000), but Oracle sizes itsNLSSORTexpression atRAW(2058)underEXTENDED; the column now takes its derived type. A schema stopped at that statement resumes with the corrected statement, and a completed 1.1.0 schema stays valid.
1.1.0 — 2026-10-07
Section titled “1.1.0 — 2026-10-07”Threadmill 1.1.0 adds an Oracle Database backend. It is a compatible minor release: core, the PostgreSQL, Redis and in-memory stores and their storage formats are unchanged, so upgrading from 1.0.x is a version bump. See Upgrade from 1.0.0.
- Added
threadmill-store-oracle, anOracleJobStorefor Oracle Database 19c and later (issue #137). It passes the same shared contract suite as the other backends and requires theAL32UTF8database character set and the defaultNLS_COMP=BINARY, both checked at startup. It uses standard JDBC only; applications supply the Oracle driver. The schema user needsCREATE TABLE,CREATE TRIGGERand a tablespace quota. See Oracle schema. - Added
OracleMigrationRunner. Oracle commits each DDL statement, so the runner records progress per statement and resumes an interrupted migration where it stopped. Concurrent migrators are serialized. - Added Spring Boot auto-configuration for the Oracle store, including
join_transactionenqueue. With both JDBC stores on the classpath, Spring picks one fromthreadmill.store.jdbc-type, then thespring.datasource.urlscheme, and otherwise keeps PostgreSQL. Oracle schema handling is set withthreadmill.store.oracle.schema-mode. - Oracle has no cross-node wake channel: a job enqueued on one node is picked up
by another within
pollInterval(default 500 ms). - The shared store contract now requires that a
nullmutex release holder never frees another holder’s mutex. - Upgraded Jackson 2 to 2.21.7 and constrained Spring Boot’s test-scope Jackson 3 to 3.1.7, resolving GHSA-7hhh-6rmp-j9qf, GHSA-p6pp-m3f8-5c89, GHSA-cxp5-3px4-pw24, GHSA-wv8q-qhhj-9h54, GHSA-gx83-3vf8-gh7j, GHSA-q4xh-88c3-wmh7 and GHSA-wjgm-6hv5-3cvf. Applications that manage Jackson through their own BOM should move to the same patched releases.
- Moved the dashboard UI build to Tailwind CSS 4.3.3 through the
@tailwindcss/viteplugin, replacing Tailwind 3, PostCSS config and Autoprefixer. Tailwind 3 resolvedbraces(GHSA-vfj7-8cjw-p6xm, no patched release) throughfast-glob,micromatchandchokidar, plus a vulnerablepostcss-selector-parser(GHSA-rj75-hqrm-r3gf); Tailwind 4 has neither dependency.source-map-jsmoves to 1.2.2 (GHSA-68fv-2mgg-jv7q).npm auditreports zero known vulnerabilities. The theme moves intosrc/styles.css, with a small base layer that keeps Tailwind 3’s border, placeholder and button cursor defaults. The packaged console now targets the browsers Tailwind 4 supports (Safari 16.4+, Chrome 111+, Firefox 128+).
1.0.0 — 2026-09-13
Section titled “1.0.0 — 2026-09-13”Threadmill 1.0.0 adds production hardening, operational tooling and migration support across the PostgreSQL, Redis and in-memory backends. See the compatibility and upgrade guide before upgrading from 0.3.0. Delivery remains at least once; handlers must be idempotent.
- Hardened execution persistence through datastore outages. Finished handlers retain a finalizer until their outcome commits or shutdown transfers recovery to another node. Heartbeats renew only confirmed active attempts, so a lost claim acknowledgement cannot leave an abandoned job permanently shielded.
- Persisted retry decisions and attempt-local execution revisions. Recovery retains the original retry policy, while delayed progress/log/check-in writes cannot overwrite acknowledged newer diagnostics or liveness.
- Fixed concurrency-hold release for unclaimed jobs and execution cleanup across timeout, shutdown and orphan recovery. Metrics and tracing distinguish overlapping contexts for the same job and unwind their resources on every exit.
- Bounded atomic bulk inserts and deferred Spring enqueues to 1,000 jobs and 8 MiB of encoded bodies. New JSON jobs reserve lifecycle space; bounded diagnostic compaction preserves identity, payload and execution policy.
- Made deferred enqueue failures observable through
AfterCommitEnqueueFailure. The defaultafter_commitmode does not make business and job writes atomic; use PostgreSQLjoin_transactionwith the same DataSource or a durable outbox. - Bounded maintenance and retention scans with resumable cursors. Retention protects pending/unknown retries, waiting workflows and live deduplication; idle queue and concurrency metadata can be reclaimed safely under churn.
- Added PostgreSQL migrations V7–V11 for execution revisions, maintenance and retention indexes, and sharded queue monitoring counters. Historical V1–V6 migrations remain unchanged. Monitoring and queue discovery use bounded or indexed operations instead of repeatedly scanning all retained jobs.
- Fixed Redis claim locks surviving timed-out/interrupted acquisitions, partial bulk acquisition and failed workflow preparation. Token-checked cleanup preserves replacement ownership and caller interruption, and tries remaining locks after an ordinary cleanup failure. Expiry remains the crash fallback.
- Raised the Redis data-node minimum to 7.4 and added an offline format-2 index migration for existing namespaces. Script-cache recovery uses key-routed EVALSHA/EVAL, including after Cluster promotion and live slot migration. Standalone, Sentinel and Cluster contracts use real Redis; failover does not imply zero acknowledged-write loss.
- Hardened application-classloader handling, dashboard authentication/error responses, Redis search capability enforcement, and optional asynchronous metrics refresh. Updated locked dependencies, including the patched Vitest development toolchain.
- Expanded regression, migration, topology and production validation. The soak harness records live invariants, operation latency, retention and fault evidence, and recovers uncertain producer acknowledgements across datastore pauses/restarts without changing job IDs.
- Updated installation and migration documentation for 1.0.0. Commercial support starts with 1.0 through hemju.com, with inquiries to sales@hemju.com. LingoHub is a reference customer.
- Removed runtime backward-compatibility layers. Persisted handler and payload
type tags now resolve exact current class names;
TypeNameAliases,PayloadMigration,PayloadMigrations,JobSerializers, andJobDefinitionMigratorare gone. The JSON serializer no longer accepts unknown payload fields. Drain or rewrite affected durable work before renaming executable types or changing payload shapes. - Removed deprecated and transitional APIs: the test-support
ForwardingJobStorealias,ThreadmillMetrics.recordClaimLatency, the timestamp-taking Redis queue-score overload, legacy Sentinel credential and TLS verification properties/constructors, shortenedJobStoreCapabilitiesconstruction, and fingerprint-free recurring-state factories. The duplicate engine-side scoped-value constant is also gone; useJobExecutionContexts.CURRENT.DashboardApiServicerequires an explicit definition validator. Spring dashboard validation now uses exact handler and payload names. - Removed Redis v0.2.1 queue-score and per-queue-age-index upgrade machinery, layout markers, node layout heartbeats, repair scripts, and mixed-version detection. Redis deployments must already use the current priority score and age index before starting 0.3.0; mixed-version rolling operation is not supported.
- Made the current contracts explicit instead of silently supplying fallback
behavior. Every
JobStorecapability, every PostgreSQL transaction-boundary capability, and engine-backedJobExecutionContextdeadline, cancellation, check-in, and result operation must be implemented. A missing operation is now a compile failure. JobStoredecorators now share the coreForwardingJobStorebase (issue #131). The OpenTelemetry and Micrometer decorators forward every SPI operation, including external-transaction and remote-wake capabilities. The reflectiveJobStoreDecoratorContractfails when any current or future operation does not reach the delegate.
- Process-fatal JVM errors are no longer swallowed as ordinary job or loop
failures (issue #101). Every intentional
catch (Throwable)now classifiesVirtualMachineError/ThreadDeath— including through bounded wrapper and cause chains — and rethrows it before logging, interceptor invocation, failure serialization, or retry work. Long-lived dispatcher, maintenance, registry, and recurring loops catchRuntimeExceptionrather thanThrowable, worker tasks run throughExecutorService.executeso a fatal reaches the thread’s uncaught-exception path instead of an ignoredFuture, and the per-job watchdog is cancelled before the fatal escapes and before terminal finalization begins. A handlerAssertionErrorremains an ordinary per-job failure. Threadmill still does not callSystem.exit: configure-XX:+ExitOnOutOfMemoryError(or an equivalent uncaught-exception handler) plus a process supervisor, because until the process stops, a fatal worker’s job staysPROCESSINGand holds its concurrency slot. - Micrometer store gauges now refresh on their own bounded pull path instead of
only on job completion (issue #100), so queue depths, per-state counts, and
oldest-age gauges stay truthful when processing stalls and when a queue is
paused or newly created. A failed refresh retains the last successful
snapshot, keeps ages advancing, and exposes
threadmill.metrics.snapshot.stale, a snapshot age, and a refresh-error counter rather than reporting zero. Concurrent gauge readers use the cached snapshot instead of queueing behind an in-flight refresh.meteredStore()now records claim latency and failures, persisted orphan reclaims, and rejected store-write attempts at the realJobStoreboundary — contractual stale-version, oversize, and invalid-argument rejections are excluded from store-health signals — and per-queue meters are capped by a configurable hard tag limit (default 100) that reports overflow throughthreadmill.metrics.queue.tags.omitted. - Fixed PostgreSQL keyed claim polling permanently re-reading the same first page of concurrency-blocked keys, which starved claimable work behind a hot key (issue #92). Queue-scoped keys are now keyset-paged from a bounded, process-local cursor with least-recently-used eviction, cursor progress survives concurrent polls through identity-checked updates, the common path still costs one loose-scan query, and a drained tail restarts within the same poll instead of reporting an empty one.
- Release publication is now fail-closed (issue #99):
productionCheckcleans and checks every subproject from one ordered task graph, supported aggregate Central Portal publication requires that gate, direct per-module publication is rejected, and a tag is rejected before the expensive build unless it exactly matches the effective non-SNAPSHOT versions of all published modules. - Hardened dashboard executable-definition edits (issue #95). Pending-job and
recurring-task updates that supply a handler or payload now require
ADMIN, pass through a host-awareDashboardJobDefinitionValidator, resolve the same handler aliases as the runtime, reject direct or inherited raw and incompatibleJobHandlerdeclarations, and deserialize the payload before any store write. This is an intentional authorization change:REPLACE_JOBandUPDATE_RECURRINGstill permit operational queue, priority, schedule, trigger, policy, zone, and enabled-state edits, but no longer select executable classes. Spring securely returns 501 for definition edits when it cannot select oneJobSerializeror an unambiguous alias registry; ordinary edits remain available. Job replacement also preserves existing producer-deduplication metadata when only the executable definition changes. - Dependency vulnerability checks now run as a distinct pull-request check, a
nightly
mainscan, and a fail-closed pre-publish release step. Each path uses pinned Node/npm plus a checksum-verified OSV Scanner and evaluates every Git-tracked Gradle/npm lockfile. The policy, advisory-specific reachability record, and time-bounded exception rules are documented. - Patched the remaining locked dependency advisories: Spring Boot 4.0.8 with
Spring Framework 7.0.9, Spring Security 7.0.7, Logback 1.5.38, and a Tomcat
11.0.25 test floor; Lettuce 6.8.2.RELEASE with Netty 4.1.137.Final;
Browserslist 4.28.9; and PostCSS Selector Parser 6.1.4. Lettuce 6.8 adds
netty-resolver-dns/netty-codec-dnsto the published Redis module’s runtime graph; these are upstream compile dependencies, not optional extras. - Redis Cluster and Sentinel now support production ACL/TLS configuration
(issue #93). Cluster credentials and verified TLS apply to every seed;
Sentinel keeps Redis data-node credentials separate from Sentinel
control-plane credentials while using Lettuce’s shared TLS policy for both.
Spring exposes the matching properties, full peer verification defaults on,
CA-only verification is also supported, and
caller-managed
RedisClusterClientinstances can be injected for custom TLS resources, mutual TLS, and rotating credentials. Topology descriptions and configuration-owned startup failures redact usernames and passwords while retaining a sanitized exception-type chain. Real authenticated/mTLS Cluster and Sentinel tests cover the injected-client and store paths. - Fixed a release-blocking Redis Cluster
CROSSSLOTfailure in ordinary job insertion and transitions. Optional LuaKEYSentries used empty strings, which hash outside the engine slot; absent keys now use the reserved{threadmill}:no_keysame-slot sentinel loaded from one Java constant. The complete shared job-store contract now runs through an actual Cluster client and asserts that the protocol sentinel is never materialized as a Redis key. - Handlers can see the engine’s deadline coming (issue #119).
JobExecutionContextgainsdeadline()andremaining(), computed by the same rule the timeout watchdog uses —claimedAtplus the effective timeout before the first check-in, the most recent check-in plusnoProgressTimeoutafter it — and capped at the end of the node’sshutdownGracePeriodonceclose()begins, so a handler that runs a loop of costly steps can stop before the interrupt instead of being cut off mid-step by a timeout or a rolling deploy.cancellation()/isCancelled()report why the engine abandoned an attempt (TIMEOUTorSHUTDOWN); the record is written immediately before the worker thread is interrupted and never cleared.JobExecutionContext.current()resolves the running context from code below the handler that has noctxparameter. All additions are default methods; existing implementations keep compiling. - The failure path now classifies a shutdown interrupt from the engine’s own
cancellation record rather than from the exception type. A handler
interrupted inside socket I/O on a virtual thread surfaces
SocketException: Closed by interrupt, notInterruptedException, and was previously billed as a handler fault — an attempt burned and the retry delayed by backoff. It is nowSHUTDOWN: rescheduled immediately, no attempt consumed. - Documented the interrupt contract on
@Job(timeout),threadmill.jobTimeout,threadmill.noProgressTimeout, and in the handler and long-running-job guides: the worker thread is interrupted; on virtual threads that can abort in-flight socket I/O (guaranteed forjava.net.Socketand interruptible channels) and leaves the interrupt flag set; a timeout interrupt is re-asserted every watchdog tick until the handler returns while a shutdown interrupt is delivered once; a handler must treat it as cancellation. - Hardened the timeout path around the new API: the watchdog keeps
interrupting a cancelled attempt even if cleanup code calls
checkIn(), a worker that registers during the shutdown sweep still classifies its interrupt asSHUTDOWN, and a per-job timeout override beyondProcessingNodeConfig.MAX_TIMEOUT(one hundred years) degrades to the global timeout instead of overflowing before the handler runs. The config rejectsjobTimeout,noProgressTimeout, andshutdownGracePeriodabove that bound. - Queue claim order is now exactly
(priority DESC, job id)on every store (issue #91). Redis queue and unkeyed-lane ZSET scores are exact negated priorities, so a lower-priority job can no longer overtake after about 116 days and extreme priorities lose no timestamp precision;JobIdnow defines allocation-free unsigned UUID natural order, matching Redis and PostgreSQL even across Java’s signedUUID.compareToboundary. Redis upgrades legacy scores in bounded, atomic pages that cannot resurrect a concurrent claim or overwrite a replacement. The layout staysrescoredwhile a legacy-scoring node heartbeat is live, then a cluster-wide lease-backed mutex elects one node for the final exact pass before it becomespriority_only_v1; rolling upgrades may temporarily perturb ordering but do not lose or duplicate jobs. The deprecatedRedisKeys.queueScore(int, long)compatibility overload is marked for removal; enqueue time is ignored. - Redis
oldestEnqueuedAt— thethreadmill.queue.oldest.enqueued.agegauge and the dashboard queue view — is now one head read of a per-queue age index ({threadmill}:queue_enqueued_at:{queue}, scored bycurrent_state_at) instead of a Lua scan that did oneHGETper ENQUEUED member inside one atomic call on every metrics scrape. The index is maintained inside the same atomic scripts as queue membership. A store started against data written by v0.2.1 or earlier backfills it once from the queue ZSETs (bounded SSCAN / ZSCAN pages from Java) and records the{threadmill}:layout:queue_enqueued_atstate:backfilledwhile any old-release node heartbeat is live (every start reconciles exactly),completeonce none remains (finalized from a read, so the last old node’s exit needs no restart; later starts check two ZCARDs per queue). New-release nodes write{threadmill}:node:layout:{nodeId}next to their heartbeat for that detection. The read verifies its head against the queue ZSET and drops a stale head through an atomic compare-and-remove. Upgrade procedure and mixed-version guarantees are in the Redis README.
0.2.2 and 0.2.3
Section titled “0.2.2 and 0.2.3”Tagged but never published. Both release builds failed in the pre-publish gate — first on a missing browser in the release workflow, then on the correctness simulation’s drain deadline expiring on a loaded CI runner — so no artifact was signed or uploaded for either version and neither exists on Maven Central. Their contents shipped as 0.2.4.
- Fixed Postgres self-owned writes being silently rolled back when the host
DataSourcehands out connections withautoCommit=false(issue #111). Queue pauses, execution and node heartbeats, maintenance leases, retention, mutexes, recurring definitions/state, and the expired-dedup fallback now use explicit Threadmill-owned transactions, restoring the connection’s prior auto-commit mode after commit or rollback.saveAtomic,softDelete,claimReady, andreplaceJobnow use that same boundary instead of forcing auto-commit on return. Migration history bootstrap and destructive schema reset also commit explicitly. The expired-dedup fallback now keeps itsSELECT … FOR UPDATElock through the delete, closing a lost-update window that also existed on auto-commit pools. - The recurring materializer now repairs a timing-fingerprint mismatch found
during its under-mutex definition reload (issue #112). The current task
definition wins: Threadmill preserves run bookkeeping and pending nudge
demand, recomputes timing forward from the tick, and does not fire or
CATCH_UPthe obsolete trigger’s overdue backlog. A listed fingerprint mismatch now triggers that repair even when the stalenextRunAtis still in the future, so a faster replacement schedule cannot remain dormant until the obsolete fire time. Legacy null fingerprints are adopted without dropping or moving an existing fire, and detected crashed edits emit an operator-visible warning. - Added fixed process-separated nudge simulations for Postgres and Redis
(issue #114). They hard-kill a maintenance leader after an accepted nudge
and prove the standby serves it, then hard-kill a producer after its durable
work write but before its nudge and prove the regular recurring backstop
drains the row. Cross-process JSON-lines traces record process ids,
leadership, trigger origins, and the verified event ordering. Ready markers
are atomically published and trace writes drain their buffers fully; the
fixed real-backend simulation is part of
productionCheck. The failover phase uses a one-minute backstop and five-minute leader poll, then edits the task to an eight-second backstop for the producer-crash phase, eliminating process-start timing races. Simulation work uses its own datastore namespace and long-lived per-process connections. - Recorded the issue #113 decision not to add per-task lifecycle generations
to the
JobStoreSPI. Issue #114 proves hard-kill handoff and backstop recovery, while the remaining revision-reuse ABA additionally requires an old materializer to resume after outliving its mutex lease and a same-name delete/re-registration. A persistent generation would not replace the existing lifecycle orderings and does not justify its cross-backend storage, migration, and SPI cost without evidence that sequence occurs. - Recurring tasks can declare claim-time exclusivity (issue #110). A
CronTaskgains anexclusiveflag, surfaced as@Recurring(exclusive = true)and as anexclusiveparameter onScheduler.defineRecurring. Every materialized instance — scheduled, caught-up, or manually triggered from the dashboard — is claimed under the derived keyrecurring:<name>inConcurrencyMode.EXCLUSIVE, so the store refuses to admit a second instance while one is processing. This replaces the per-handler advisory lock a singleton sweep usually grows. It does not close the lease-expiry reclaim window: reclaim releases the concurrency slot as part of the terminal failure save, so handlers stay idempotent by contract. Postgres adds the additive migrationV6__cron_task_exclusive.sql; Redis stores the flag as a cron-task hash field with the existing overwrite-on-re-upsert semantics. - The recurring pile-up guard no longer opens a window between a failed
instance and its retry (issue #110).
FAILEDis not terminal — a retry may follow — but the guard treated everyFAILEDas finished, so a materializer tick landing between the failure save andRetryInterceptor’s reschedule save could create a fresh instance beside a retrying one. AFAILEDinstance now blocks while its retry budget is not provably spent and the failure is younger than a five-second handoff grace. A retry-exhausted instance still never blocks, and the age bound keeps an instance that is terminal under a per-exception-type policy — which the guard cannot read — from blocking its task until the stranded-failure recovery scan reaches it. DashboardPayloads.CronTaskViewgains anexclusivecomponent.- Documented the three overlap windows and the reclaim window’s unclosability, with fencing guidance for effects that must not happen twice (issue #110).
- On-demand materialization for recurring tasks (“nudge”, issue #108).
Scheduler.nudgeRecurring(taskName)(Spring:JobScheduler.nudgeRecurring) requests that a registered recurring task materialize an instance as soon as possible, turning frequent pollers into wake-driven pumps with a slow backstop schedule. The nudge goes through the normal machinery — same pile-up guard, same missed-run policy — with four guarantees: at least one run starts after every accepted nudge (a nudge during an in-flight run produces exactly one follow-up after it completes, because the in-flight run may have read its inputs before the nudge’s triggering write committed); a burst of nudges coalesces to at most the current run plus one follow-up; the nudge is a durable store write consumed by the maintenance tick, so there is no transient signal to lose (worst-case latency onemaintenancePollInterval, default 1 s); and a nudged run never moves the schedule — a cron task’s next fire stays the regular wall-clock match and an interval trigger’s phase is preserved. Nudging an unknown task throws; nudging a disabled task throws — an explicit pause wins — and an enabled-flip clears any pending nudge. The coalescing bound is failure-free: consistent with at-least-once, a crash between the follow-up’s insert and the request’s clear can produce an extra run, never lose one. Storage: additive Postgres migrationV5__cron_state_nudge.sqladdsthreadmill_cron_task_state.nudge_requested_atplus a store-generated, never-resetnudge_revision— the revision, not the collision-prone wall-clock timestamp, is the compare-and-clear identity; Redis stores both as schedule-state hash fields.CronTaskScheduleStategained read-onlynudgeRequestedAt/nudgeRevisioncomponents. - Spring nudges are after-commit in every enqueue mode, including
join_transaction. Validation fails fast at call time, the write lands on commit, and a rollback discards it — identical semantics in all three modes. This is the one write that deliberately does not join the caller’s transaction: coalescing is one store cell per task, so a joined nudge would hold that row’s write lock for the whole business transaction and serialize every concurrent producer of that task (silently — correct at low rate, collapsing under load), and all it would buy is closing a crash window the design explicitly does not need closed. An in-JVM per-task coalescer additionally bounds the store write rate under bursts, and never retains a caller beyond its own covering write (follow-up generations run on a dedicated virtual thread). - Spring: nudge by handler class,
jobScheduler.nudgeRecurring(OutboxPump.class). A@Recurringtask’s durable identity defaults to the handler’s fully-qualified class name, so the string overload forced callers to hard-code it and broke on a rename or package move; the class overload resolves the registered name through the handler registry and matches the rest of this API, where the handler class is always the first argument. Nudging a registered handler that is not@Recurringfails loudly. The string overload remains for tasks registered imperatively through the coreScheduler, where the caller chooses the name. - Breaking (SPI):
JobStoregained two abstract operations,requestCronNudge(name, requestedAt)→ACCEPTED | UNKNOWN_TASK | DISABLEDandclearCronNudge(name, observedRevision). Third-party store implementations must add both before recompiling (old binaries throwAbstractMethodErrorwhen nudged). Implementation contract: acceptance must atomically check task existence + enabled and advance a monotonic, never-reset revision; the clear must compare-and-clear on that revision and must not touch it;upsertCronTaskStatemust preserve both nudge fields; a nudge must never resurrect state for a deleted task. The contract test suite (AbstractJobStoreContractTest) pins all of this. - Recurring instances now carry
threadmill.cron.originmetadata (schedule/nudge/manual), surfaced in three places: handler code (JobExecutionContext.cronOrigin()), the dashboard API (JobSummary.cronOrigin, deliberately visible on redacted read-level views — the value set is closed, so no metadata can leak through it), and Micrometer (threadmill.jobs.recurring.runs{origin=schedule|nudge|manual|other}, cardinality-clamped). The operations console renders it as a badge beside the handler, so a nudged run is distinguishable from a scheduled one at a glance.
- Breaking (Spring): renamed
@Job(maxRetries)to@Job(maxAttempts)(issue #104). The value always counted total attempts —maxRetries = 1meant one attempt and zero retries — so every existing number keeps its runtime meaning under the honest name; only the attribute name changes. Non-positive values other than the-1sentinel now fail startup with a descriptive error instead of being silently replaced by the engine default: the natural “disable retries” spelling0used to run up to five attempts. The rename also reachesThreadmillJobRegistry.Registration: its component changes fromint maxRetriesto a nullableInteger maxAttempts(constructor, accessor, and binary signature change;nullmeans “no explicit budget”). Code that constructs registrations directly or reads them viaThreadmillRecurringRegistrar.recurring()must follow the new shape. - Per-job retry metadata is now stamped only for an explicit
@Job(maxAttempts)(issue #104). Previously the resolved default was stamped on every Spring-enqueued job, so per-job metadata permanently shadowed per-exception-typeRetryInterceptorpolicies. Jobs without an explicit budget now fall through to the policy chain, and@Recurringhandlers without an explicit budget register theirCronTaskwith a null retry budget, so laterthreadmill.default-max-attemptschanges are honored by already-registered tasks. - Fixed startup re-registration wiping an overdue recurring
next_run_at(issue #105).Scheduler.upsertCronnow preserves the schedule state — including an overdue next-run and an interval trigger’s phase — while the re-registered schedule is unchanged, so the task’sMissedRunPolicydecides what happens to restart-missed firings:CATCH_UPmaterialises every missed fire (capped per tick) andDROPcollapses the backlog into one make-up run shortly after startup. A real edit — changed trigger, changed zone on a cron trigger, or re-enabling a disabled task — still recomputes the next-run from now so an edited cron never fires stale times.CronExpressiongained source-based value equality to support the unchanged-schedule comparison. CronTaskScheduleStatenow carries atimingFingerprintrecording which trigger timing itsnext_run_atwas computed from, written atomically with the state row (additive Postgres migrationV4__cron_state_timing_fingerprint.sql; a Redis hash field; legacy rows read as null and adopt the current fingerprint without moving their recorded timing). The unchanged-schedule decision reads this fingerprint rather than comparing stored task definitions, so a crash between the separate task-definition and state writes can never pair a new trigger with old timing undetectably — the retry detects the mismatch and recomputes. The fingerprint deliberately excludes the zone for interval triggers (CronTaskdocuments the zone as ignored there), so nodes with different system-default zones do not treat the same interval as a schedule edit.DROPmissed-run recovery is now phase-exact and nominally stamped: the single make-up instance represents the most recent nominal fire (itscronFireTime()carries that nominal time, not the recovery wall-clock), and the next run advances from the nominal fire — an every-6h task that was due at 06:00 and recovered at 07:00 next fires at 12:00, not 13:00. Behavioral note: with overdue state now surviving restarts, an existingDROPtask whose boundary passed during downtime fires once shortly after startup instead of silently skipping to the next boundary — the same behavior it always had across a live materializer stall.- Patched OSV-flagged dependencies for the release: PostgreSQL JDBC 42.7.12,
Netty 4.1.136.Final, Micrometer 1.15.12, a test-classpath constraint on
tools.jackson.core:jackson-databind3.1.5 (Spring Boot 4.0.7’s BOM still pins 3.1.4), and refreshed dashboard build dependencies (nanoid,postcss).npm auditand the OSV scan report zero known vulnerabilities for the release lockfiles.
- Fixed Spring lifecycle ordering so
ProcessingNodestarts before remote-wake subscription and stops after the subscription closes. Spring applications now fail fast when no durable store is configured; the in-memory store must be explicitly enabled withthreadmill.store.memory.enabled=true. - Required the per-task correctness mutex for recurring definition registration, reconciliation, dashboard mutation, and deletion. Contended mutations now fail explicitly instead of proceeding without their lock.
- Retained terminal-save responsibility through store outages. A worker keeps
retrying
PROCESSING -> SUCCEEDED/FAILEDwith capped backoff until the write commits or node shutdown begins, preventing completed attempts from being stranded behind refreshed owner heartbeats. - Hardened persisted invariants with defensive
JobSnapshotcopies, bounded handler type names, safeProcessingNodelifecycle behavior, startup migration checksum/description validation, and additive PostgreSQL V3 integrity constraints. - Bounded Redis claim candidate discovery with rotating
HSCANcursors and pipelined per-key probes, avoiding backlog-wide registry reads under large keyed workloads. - Added Gradle wrapper verification, dependency locking and SHA-256 metadata,
immutable GitHub Action pins, and mandatory
META-INF/LICENSE/NOTICEinspection for every published JAR. Refreshed dashboard build tooling to patched Babel, esbuild, and Vite releases;npm auditreports zero known vulnerabilities for the release lockfile. - Upgraded Jackson, Spring Boot, Spring Framework, Spring Security, Netty, PostgreSQL JDBC, Logback, and AssertJ to patched maintenance releases. The release OSV gate now scans every committed Gradle and npm lockfile instead of passing an unsupported version-catalog file to the scanner.
- Synchronized installation versions, release/security guidance, API names, dashboard security documentation, Redis execution-model documentation, and soak-result wording.
- Fixed
@Job(timeout)and@Job(maxRetries)being silently ignored for@Recurringhandlers (issue #84). ACronTasknow carries an optional per-instancetimeoutandmaxAttempts;RecurringMaterializerand the dashboard’s manual trigger stamp them onto every materialized instance asJobRunner.META_TIMEOUT_SECONDS/RetryInterceptor.META_MAX_ATTEMPTS, so the annotations behave identically on the recurring and enqueue paths. Previously every recurring instance ran under the globaljobTimeoutand the default retry budget, so a long-running recurring job could be interrupted mid-run and retried (duplicating external side effects). - Added a per-instance
timeout/maxAttemptsparameter toScheduler.defineCronTask,defineIntervalTask, anddefineRecurringfor core (non-Spring) callers. Existing signatures are unchanged (nullkeeps the engine defaults). - Added the additive Postgres migration
V2__cron_task_overrides.sql(nullabletimeout_secondsandmax_attemptsonthreadmill_cron_tasks); existing rows default to the engine behaviour. Redis stores the same as optional hash fields cleared by an override-less re-registration.
- Fixed
Dispatcherrelease of claimed-but-unrun jobs (node-tag mismatch, dispatch failure, shutdown mid-batch): releases now route through the single failure path asFailureCause.SHUTDOWN, so the job is rescheduled immediately without consuming a retry attempt and its claim-time concurrency slot is freed. Previously the release attempted an illegal PROCESSING→SCHEDULED transition, always threw, and left the job to orphan reclaim. - Fixed the default job id and
createdAtto derive from a single clock read inJob.Builder.build(), keeping UUIDv7 id ordering consistent with the engine’s in-key claim-admission order. - Validated with a 12-hour PostgreSQL endurance soak (mixed workload, ~5.5 million jobs, 3 nodes, node churn every 10 minutes): passed with all invariants green.
First public release under the Apache-2.0 license.
- Added claim-time per-key concurrency with
ConcurrencyMode, workflow-root inheritance, store-backed enforcement in memory/Postgres/Redis, and documentation for import/export and tenant event-processing shapes. - Added queue-family lanes with anchored
*/?patterns, stride-scheduledQueueWeights, discovery retention, Spring configuration, and soak coverage across all stores. - Fixed workflow-root concurrency release for failed intermediate workflow steps by abandoning descendants that can no longer be promoted.
- Added the Spring ergonomic API:
@Job, typed handler discovery, andJobScheduler. - Added Redis standalone, Sentinel, and Cluster configuration. Cluster uses a
single
{threadmill}hash slot for v1 Lua correctness. - Added producer-side deduplication with
Created/Coalescedresults. - Added long-running job check-ins, progress updates, bounded logs, and no-progress timeout handling.
- Removed the experimental alternate framework module and public positioning for now. The core remains framework-agnostic so additional integrations can be added later.