Pausing the Feature Annotation Tool

Home / Engineering / Decisions / Pausing the Feature Annotation Tool

Engineering Log Entry — April 2026

Decision to pause development of the feature annotation tool in favor of exploring a new CAM strategy based on B-Rep primitive surfaces. The tool is preserved for potential resumption.

Key resources:

• Annotation Tool — Linear Project
• Annotation Data Schema Alignment — Google Doc

Context

We have been building a feature annotation tool for labeling 3D CAD models with manufacturing features (pockets, holes, slots, chamfers, etc.) and GD&T annotations. The goal was to use offshore annotators to label a large dataset, then train ML models for automatic feature recognition — a core input to the AutoCAM pipeline.

The tool is substantially built. It started as a standalone React + FastAPI application (annotation_tool_web) with a Three.js 3D viewer, pythonocc STEP processing, and integration with three ML detectors (AAGNet, BrepMFR, BRepFormer). It supports 37 feature types, GD&T annotations, instance grouping, and export to multiple formats. The tool has been mostly migrated into Carbon (our manufacturing platform monorepo), backed by Supabase with auth and permission controls. Junaid Khan led the integration work.

Significant design work was also completed: a dataset annotation schema (derived geometry, AAG graph, feature labels, GD&T/PMI extraction, part-level screening labels), a two-layer feature taxonomy aligning ML and CAM use cases, and a multi-annotator workflow with role-based access, part claiming, and ML pre-labeling.

However, no annotated data has been produced yet. The tool was nearing deployment readiness — Junaid was finishing polish features (auto-save, role-based permissions, drawing uploads) — but annotators had not yet started labeling.

What Changed

Kelsey proposed a new CAM planning strategy that works from B-Rep primitive surfaces rather than higher-level manufacturing features. Instead of recognizing features like "blind pocket" or "through hole" and assigning toolpaths to those features, the approach:

1. Detects primitive surface types directly from the B-Rep (ruled surfaces, ball sweeps, freeform surfaces)
2. Matches tool profiles to surface geometry (e.g., the side of an endmill lies on a ruled surface, a ball endmill fits a concave spherical surface)
3. Generates finishing toolpaths per surface-direction pair in parallel
4. Selects an optimal subset of setups that covers all surfaces (a set cover problem)
5. Generates roughing toolpaths per setup, then post-processes into a rest-roughing sequence

The key difference: this approach operates on geometric primitives that OpenCascade already exposes from the B-Rep, rather than on a learned feature taxonomy that requires training data. It sidesteps the need for a labeled dataset and the annotation pipeline entirely.

This is an approach without much precedent in industry — most CAM systems are feature-based. But we are confident in Kelsey's skillset and want to allocate our small team (4 engineers) to exploring this direction. Early results show partial finishing on one or two parts; the rest of the approach is on paper and being fleshed out.

Decision

Pause development of the feature annotation tool. Preserve the codebase and infrastructure for potential resumption.

Immediate scope:

• Junaid finishes in-flight tasks that are near completion: auto-save, role-based permissions, drawing upload support
• Junaid stops short of heavy new work: no backend architecture redesign, no AAG integration, no new design documents
• After reaching a natural stopping point, Junaid pivots to other platform work (customer portal integration with ERP/MES)
• The Linear project gets a note explaining the pause; remaining issues move to backlog
• GD&T annotation, when needed, will be handled by offshore annotators using tools they already know (OnShape, SolidWorks) rather than the custom annotation tool

What we are not doing:

• We are not removing the tool from Carbon or deleting infrastructure
• We are not making a permanent decision about feature-based ML — this is a resource allocation pivot, not a technical conclusion

Reasoning

Why pause, not continue

We are a team of four. The annotation tool's primary purpose — producing training data for feature recognition ML — is no longer on the critical path if Kelsey's primitive-surface approach works. Continuing to invest in annotation tooling, offshore annotator onboarding, and dataset labeling while simultaneously exploring a fundamentally different CAM strategy would split our limited resources across two divergent bets.

No annotated data has been produced yet. The tool is built but the expensive part — actually running a labeling operation — hasn't started. Pausing now costs us very little compared to pausing after we've hired and onboarded annotators.

Why pause, not kill

The primitive-surface approach is unproven. If it doesn't scale — particularly for complex multi-setup parts where setup selection becomes intractable, or geometries where primitive surface matching isn't sufficient — we would fall back to a feature-based approach. The annotation tool would be the starting point for that fallback, and the integration work Junaid has done would not need to be repeated.

Feature recognition may also have strategic value beyond CAM toolpath generation. For example, semantic feature information could power automatic DFM feedback to customers ("this pocket requires a tool change" or "this undercut is not machinable in 3-axis"). That use case doesn't justify continued investment today, but it's a reason not to throw the work away.

Why this is a reasonable bet

The primitive-surface approach has a structural advantage: it works from geometry the CAD kernel already provides rather than requiring a trained ML model. If it works, it eliminates the entire annotation → training → inference pipeline and the ongoing cost of maintaining labeled data. The tradeoff is that it's less proven and currently single-threaded through Kelsey.

What We Preserve

Asset	Status	Notes
Annotation tool in Carbon (apps/annotation-tool)	Code complete, not deployed to annotators	Three.js viewer, 37 feature types, GD&T, instance grouping, undo/redo
Supabase schema and infrastructure	Running	annotationSession, annotationDocument tables; auth and RLS policies
Test data in Supabase/S3	Present	STEP files and test sessions from development
Dataset annotation schema design	Documented	Feature taxonomy, derived geometry, AAG, GD&T/PMI extraction, part-level screening
Linear project and issues	Backlogged with pause note	11 issues across two projects; mix of in-review, todo, and backlog

Resumption Criteria

We will reassess in 1–2 months based on progress with the primitive-surface approach. Signals that would bring the annotation tool back:

• The primitive-surface approach fails on a significant class of geometries that feature-level reasoning would handle (e.g., complex interacting features where setup selection needs semantic context)
• We decide to pursue feature recognition as a foundational capability for non-CAM uses (DFM feedback, quoting, customer-facing part analysis)
• We adopt a hybrid strategy that uses primitive surfaces for toolpath generation but features for other pipeline stages

There is no fixed trigger — the team will re-evaluate continuously as Kelsey's approach is tested on more parts.

Discussion Topics for the Team

These are open questions to resolve in upcoming discussions:

Infrastructure cleanup

The annotation tool leaves database tables, a Carbon app directory, and test data (stored in a separate system) in place. Do we leave everything running until we have more clarity on the path forward, or clean up now? Most likely we leave things as-is, but this deserves a deliberate decision rather than drift.

Single-threaded risk on the new CAM approach

Kelsey is currently the only person who can meaningfully push the primitive-surface CAM strategy forward. Custom toolpath generation requires deep computational geometry expertise that isn't easy to jump into. Questions to discuss:

• Should we hire another computational geometry engineer to parallelize this work?
• How should David and Avnish support Kelsey — pair programming, taking on specific sub-problems, handling infrastructure so he can focus on algorithms?
• What else can we do to accelerate validation of this approach?

Validation plan

The plan is to test incrementally — one part, then two, then three, then N — looking for cases where the approach breaks down. Examples of things to watch for:

• Sequencing — how to approach toolpath ordering, part orientation, and the set cover problem for surface-to-setup assignment
• Geometries where primitive surface matching doesn't produce adequate toolpaths (e.g., complex blends, interacting features?). This ties into optimization e.g. of cut times later on.

The goal is a strong signal within 1–2 months on whether this approach will take us most of the way, or whether we need a different or hybrid strategy.