Technical evaluation · for perception / ADAS engineers

Cameras can't see it. LiDAR can't see it. We see it.

For perception engineers at automakers, Tier-1 suppliers and autonomy vendors evaluating non-line-of-sight pedestrian detection. We run an urban approach with a child hidden behind parked vehicles and demonstrate, end-to-end, 16 patent claims on a single page — 7 automotive-specific (multi-physics NLOS, AND-gated fusion, phase-invariant FFT chain entry, frame-rate sealing, OEM-installed-sensor retrofit, EDR ring-buffer integration, V2X cooperative trust) and 9 cross-cutting (verified-prior recursive smoother, cascade tamper invalidation, canonical serialization, multi-party offline verification, industry-standard format embedding, cloud-TEE PUF emulation, FPGA hardware HMAC with metering, sidecar retrofit, and — most importantly for production — fault-tolerant odd-N PUF quorum operation under natural physical degradation across a 10-year ASIL-D fleet lifetime). All numbers from a reproducible Python pipeline.

✅ Update — validated on real data: on real K-Radar (KAIST) 4D radar, sequence 20, the Axowl modal stack detected 5 of 5 radar-invisible pedestrians a median +4.6 s before conventional radar first sees them (the prior simulation had estimated +2.35 s). Next: Doppler-domain testing on the full 4D tensor — to separate a moving hidden pedestrian from static clutter. See the measurement ↓

Request engineering briefing Read the whitepaper

✅ Real-world data · K-Radar (KAIST) · validated on sequence 20

On real K-Radar data, we detect the radar-invisible pedestrian a median +4.6 s before conventional radar.

The CPNCO demo further down is a concept simulation. This result is real measured data . The Axowl modal stack runs end-to-end on real K-Radar 4D radar tensors (KAIST — K-Radar, NeurIPS 2023 Datasets & Benchmarks). K-Radar's v2.1 visibility labels tag each pedestrian by what each sensor can see: avail = L1 means LiDAR confirms a pedestrian there that conventional radar cannot detect . Those radar-invisible pedestrians — 1,042 across 988 frames — are exactly the NLOS target. On sequence 20 (urban night), the modal stack recovered 5 of 5 of them from the radar tensor a median +4.6 s before conventional radar first registers the same pedestrian (the dataset's own L1 → R visibility transition). LiDAR is the ground truth here; the claim is that we pull the pedestrian out of radar seconds before conventional radar processing catches up.

seq20 radar-invisible pedestrian track

Range when conventional radar first detects

Modal lead time (MTI)

Track 129

21.5 m

+4.8 s

Track 237

21.5 m

+5.8 s

Track 235

29.3 m

+4.6 s

Track 238

27.8 m

+2.4 s

Track 236

31.3 m

+1.5 s

Median (5 / 5 detected early)

+4.6 s

What "+4.6 s" means. The prior simulation estimated +2.35 s; the real K-Radar measurement is roughly 2× larger. Detection rule: a per-frame metric z-score ≥ 2.5 sustained over 3 consecutive labelled frames; the winning metric is m3 = MTI (zero-Doppler-masked radar energy). Azimuth sign is calibrated from avail = R objects — no geometric assumption. Reproducible: measure_head_of_time.py / find_transitions.py in scripts/demo_automotive/k_radar/ .

Watch on YouTube ↗  ·  Download MP4 (3.6 MB)

Real-data demo video (sequence 20). Six panels on the real K-Radar capture: front camera with projected 3D boxes (red = radar-invisible L1 , blue = radar-visible R ), the radar tensor, the modal signature, and the running lead-time readout. No synthetic frames — every panel is the KAIST capture.

Honest scope — three sequences, reported in full.

seq20 (urban night): 5 tracks, 77–119 radar-invisible frames each → median +4.6 s . ✅

seq16 (daytime): 3 tracks, only ~10 invisible frames each → +0.2 s . The 1-second window is a structural floor, not a failure — there is little head-room to win.

seq30 (fog / rain): 9 pedestrian tracks but no L → R transition in the capture → lead time not measurable with this method.

This is one validated sequence with a clear NLOS window — not a full-dataset NLOS mAP, and not yet the Doppler 4D test. Next step — Doppler: the full arrDREA tensor adds the Doppler axis, which separates a moving hidden pedestrian from static clutter (parked cars, walls) — the core of NLOS pedestrian safety. The seq20 result above already uses MTI (a Doppler-derived mask); full 4D Doppler is the next measurement.

Why mmWave + modal — not just another sensor

Vision + AI solves what data can solve. We solve what data cannot .

Self-driving perception splits into two kinds of tasks. Camera + ML has already solved the data-rich 99% of everyday driving — lane marking, signal colour, sign recognition, pedestrian classification, intent estimation. We do not compete with that. The remaining 1% — occluded NLOS pedestrians, heavy-fog + night combinations, edge-case anomalies — is the long-tail that no fleet of any size can collect, because the camera that would label the data cannot see what is occluded in the first place . That is the part we solve, with physics rather than data.

Data-rich · 99% of driving

Data-scarce long-tail · 1% of driving (≈ 100% of severe accidents)

Approach

Camera + ML — interpolation across abundant labels (Tesla, Mobileye, modern Tier-1)

mmWave + modal decomposition + verified-prior — extrapolation by physics forward model

Tasks

Lane marking · signal colour · traffic sign · pedestrian classification · pose / intent · daytime LOS object detection

NLOS occluded pedestrian (camera cannot label what it cannot see) · heavy fog + night + accident combination · stalled emergency vehicle behind curve · edge-case debris · pre-crash anomaly patterns

Why fleet data works

Pattern repeats every drive — labels are abundant and cheap. ML interpolation is the right tool.

Events are structurally rare — < 1 / 10⁵ km natural occurrence, plus the labelling sensor is itself blind to the event. No amount of fleet data closes this gap.

What we add

Nothing — vision + ML is the right answer. Keep the camera stack as is.

mmWave 4D imaging diffraction multipath + modal decomposition (8-coefficient phase-invariant FFT) + verified-prior smoother + PUF-anchored chain. Physics-grounded feature, OOD-generalisable, no fleet data required to operate.

Where Axowl plugs in

Untouched — existing OEM camera stack stays as production lead.

Drop-in mmWave + modal layer alongside the camera stack. Camera handles the 99%; we cover the 1% that the camera structurally cannot.

The accident-distribution observation. Public incident analyses of camera-AI driving systems (NHTSA SGO crash reports, IIHS investigations) consistently place serious crashes inside the long-tail 1%: occluded objects emerging from corners, white trailers against bright sky, stationary emergency vehicles, road debris, sensor failure under heavy weather. The structural problem is that the data needed to train these failure modes is the data the failing sensor cannot collect . A physics- driven secondary modality — one that does not depend on the same blindspots — is the only way to close the loop. mmWave diffraction multipath + modal decomposition is that modality.

Honest scope. The 99% / 1% framing is a qualitative observation about where serious failures happen , not a quantitative crash-cause attribution. Quantitative validation requires (a) public-benchmark (VoD / K-Radar) mAP measurement of our modal stack vs production baselines, (b) self-collected NLOS bench dataset with ground-truth occlusion, (c) OEM-pilot ROC / PR on real-world long-tail. (a) has started — the modal stack now runs on a real K-Radar radar tensor and the radar-invisible-pedestrian evaluation set (K-Radar v2.1 visibility labels) is identified; per-frame mAP scoring is in progress (§12). (b) and (c) remain on the roadmap. No NLOS mAP number is claimed as completed today. See §12 limitations and §14 pilot scope.

Sensor-by-task labour division — where Axowl plugs in, where the camera stack stays untouched

Perception task

Best-fit sensor (legacy stack)

Axowl stack contribution

Lane marking

Camera — unchanged

None — existing camera stack stays as production lead

Traffic-signal colour

Camera — unchanged

None — existing camera stack stays as production lead

Traffic sign (text / icon)

Camera — unchanged

None — existing camera stack stays as production lead

Pedestrian intent / gaze / pose

Camera + ML — unchanged

None — existing camera stack stays as production lead

Object classification (pedestrian / vehicle / cyclist)

Camera + ML — unchanged

None — existing camera stack stays as production lead

LOS object detection

Camera + LiDAR-replaced mmWave

Inherits LiDAR's 3D-detection role

3D depth / range

LiDAR-replaced mmWave 4D imaging

Full LiDAR replacement (4D imaging tensor + modal)

Velocity (Doppler)

mmWave — direct measurement

More accurate than camera frame-to-frame inference

Adverse weather (rain / fog / snow)

mmWave only penetrates

Axowl-unique — camera + LiDAR both fail

Night / low light

mmWave — unaffected

Complements camera low-light degradation

Long-range (200 m +)

mmWave 200–300 m

Complements camera ≈ 200 m perception limit

NLOS / occluded objects

mmWave + modal only

Axowl-unique — camera + LiDAR are 0 %

How to read this table. Five rows where the camera stack stays as production lead and Axowl adds nothing — these are tasks already solved by data-rich vision + ML (Tier-1 / Mobileye-class). Five rows where Axowl complements or replaces the LiDAR + long-range / weather / low-light role at a fraction of the BOM cost. Two rows highlighted red are Axowl-unique — physical regimes where every other production sensor is structurally blind.

⚠ Demonstration video · simulation only

Auto NLOS — 30 km/h CPNCO simulation, full pipeline visualised.

The first ~25 seconds cover what is established science — peer-reviewed mmWave NLOS literature, 40 kHz ultrasonic physics, Euro NCAP CPNCO test standard, and sensor BOM cost comparison. After the explicit "SIMULATION ONLY" transition, the 6-panel simulation core renders the deterministic seed-locked Python pipeline frame by frame: bird's-eye view, driver POV, 77 GHz mmWave tensor, 8-coefficient modal output, PUF-anchored chain seal timeline, and the head-of-time indicator (2.35 s ahead of camera / LiDAR line-of-sight). Closing slides outline the path to public-benchmark validation.

Watch on YouTube ↗  ·  Download MP4 (1.1 MB)

Watermark. Every simulation-core frame carries the "SIMULATION ONLY · NOT PRODUCTION DATA" watermark. Numbers in the video are deterministic (seed = 20260514, single trial). The video is concept demonstration; quantitative validation requires public-benchmark (VoD / K-Radar) mAP, self-collected NLOS bench, and OEM-pilot ROC / PR — see §12 / §14.

Source code. Reproducible Python pipeline at scripts/demo_automotive/automotive_video.py — single deterministic seed, matplotlib + ffmpeg renderer, no proprietary dependencies. Re-run on any workstation with Python 3.11+ and ffmpeg installed.

§0 · Reader's note — what is and is not claimed

Two independent value props on this page. Evaluate them separately.

(1) NLOS perception hypothesis (§3 – §4, §6 – §7). That a phase-invariant modal decomposition of mmWave + ultrasonic multipath carries usable NLOS signal — and that an integrity-gated verified-prior smoother (XC1) yields lower trigger-time variance with a measurable, safe fall-back to single-shot operation under chain tamper. The page demonstrates the primitive in a 2-D simulator; the production-grade numbers a Tier-1 perception team needs (false-alarm rate per hour, ROC / PR on held-out scenarios, generalisation to adult / cyclist / animal, real-world ultrasonic range, cross-vendor reproducibility) are the explicit scope of a pilot — see §12 + §14.

(2) Tamper-evident perception chain (§5, §7 XC9, §8 XC4). The FPGA-PUF sealed transition hash chain + fault-tolerant odd-N quorum + multi-party offline verification stand on their own and are independent of which perception algorithm produced M t . They would apply equally to a CNN, a transformer, a vendor's proprietary fusion, or even today's production LiDAR stack. This is the primary licensable portfolio asset.

A licensing partner who is sceptical of (1) on toy-simulator grounds can still adopt (2) on its own — the integrity primitives do not depend on the NLOS hypothesis being correct.

⚡ Signal / Data Restoration

Broken mmWave + ultrasonic → hidden child recovered.

Patent core claim: "recover original modal information from broken received signal via medium forward model + integrity-anchored prior" . Cameras and LiDAR cannot see the occluded target — diffuse mmWave and diffracted ultrasonic carry traces of it. The modal forward model + XC1 verified-prior smoother reconstructs the latent target trajectory from these broken signals. See §3 setup + §4 sensor signatures .

§1 · Why an ADAS / autonomy team should read this

Sees through occlusion. Reuses existing OEM-installed sensors. Sealed for after-action review.

A 30 km/h urban approach with a child hidden behind two parked vehicles shows three effects. First, fusing 77 GHz mmWave radar diffraction with 40 kHz ultrasonic returns extracts a modal anomaly 2.35 seconds before any line-of-sight sensor — camera, LiDAR, or otherwise — can detect the child. Second, the sensors needed are already installed on virtually every production vehicle (mmWave on ~80 % of new cars, ultrasonic parking arrays on 100 %). Third, every M t snapshot is signed inside an FPGA-resident PUF and chained — turning the perception stack's output into auditable evidence for incident review.

NLOS pedestrian detection

Children, pets, cyclists behind parked cars / corners — invisible to camera and LiDAR, audible to mmWave + ultrasonic multipath via modal decomposition.

5 – 50× cheaper than LiDAR

LiDAR stack: $1,500 – $5,000 / vehicle. RF + ultrasonic stack with our algorithm: $100 – $400 / vehicle using existing OEM-installed sensors.

After-action / incident review

Sealed perception chain replaces "we trust the EDR" with "we cryptographically verify what the car knew, and when".

Regulator-ready evidence

NHTSA, KATRI, UNECE WP.29 — the chain gives an investigator a tamper-evident perception log without trusting the OEM's logging stack.

§2 · Experimental setup

Urban approach, 30 km/h, two parked cars, hidden child.

We use a 2-D top-down model deliberately — no full vehicle-dynamics integration with CarMaker / Carla / VTD, no ASIL-D-grade sensor stack, no production radar signal processor. The point is to isolate the perception primitive (the M t -extraction of Subsystem D-2) from confounding gains from a production fusion stack, an HD map, or a behaviour planner. Porting onto a higher-fidelity simulator is the obvious next step of any pilot.

Scene

Urban side-street, ego in main lane at 30 km/h, two parked cars at x = 32 m / 37.5 m , child 1.5 m behind the second car

Ego vehicle

2-D point mass, length 4.6 m, width 1.85 m, cruise 8.3 m/s, comfortable braking 6.0 m/s² (0.6 g), 0.3 s reaction delay

RF radar (claim 4)

77 GHz mmWave, ±45° FOV, 80 m range, multipath-around-corners via wall diffraction; modeled as a 24×20 azimuth-range intensity map

Ultrasonic array (claim 12)

40 kHz, ±60° FOV, 25 m simulator ceiling, diffraction-around-corner pulse-echo amplitude (the same physics that already powers a parking sensor). Physical caveat: 40 kHz air attenuation is ≈ 1.3 dB/m at 20 °C / 50 % RH, so 25 m is the synthetic upper bound, not a measured field range. Production parking arrays typically operate to ≈ 5 m and edge-bench research arrays to ≈ 10 – 15 m; closing the gap between simulator and bench is the D-test of a pilot.

Integration step

20 Hz update, 10 Hz chain sealing

Modal extractor

8-coefficient phase-invariant FFT magnitude per modality — 16-D M t after concatenation

Fusion gate

Geometric-mean AND of noise-floor-subtracted RF and ultrasonic scores — both must rise above floor for trigger

Camera-only baseline

Sees the child the instant the LOS clears (ego at +1 m past the side-street corner) — same instant a LiDAR would, for an occluded target

§3 · Scene and detection comparison

Same scene, two cars: one stops, one hits.

Both autopilots see the identical road geometry. The camera-only autopilot has no NLOS information until line-of-sight clears the lead parked car — at which point the stopping distance (~ 8.2 m from 30 km/h with 0.3 s reaction) is insufficient. The fused autopilot triggers on the joint multipath signature 2.35 seconds earlier , leaving ~ 20 m of clearance.

Detector

First trigger (m to corner)

Clearance to impact

Outcome from 30 km/h

Camera / LiDAR — naïve LOS-only

~ 1 m

~ 1 m

collision (8.2 m needed)

Camera / LiDAR + occlusion-aware prior †

~ 6 – 8 m

~ 6 – 8 m

safe (conservative slow-down only)

RF mmWave modal

~ 12 m

~ 12 m

safe (but tight)

Ultrasonic modal

~ 10 m

~ 10 m

safe

Fused RF + ultrasonic

~ 20.5 m

~ 20.5 m

safe + 2.35 s advance warning

"Trigger distance" is measured at the first frame after a 3-frame run above the per-detector threshold. The fused detector wins not because RF or ultrasonic alone are insufficient, but because the AND-gate on the joint signature is what gives us a low-enough false-alarm rate to actually trigger braking without nuisance stops.

What this table does not measure. Trigger distance is a single-scenario metric on a fixed threshold. It is not a false-alarm rate per hour, a ROC curve, a PR curve, or an unseen-scenario generalisation test. The threshold here is tuned to this scene; a production deployment requires the threshold fixed across a held-out scenario library (rainy night / dense urban clutter / multiple-occluders / non-pedestrian targets) and the resulting curves published. That measurement is the explicit scope of a Tier-1 pilot — see §12 limitations and §14 next steps. Until those numbers exist, the claim of this table is "the NLOS primitive carries information, and the AND-gate makes it usable in principle," not "production-ready under arbitrary clutter."

† Modern LiDAR / camera baselines are not LOS-only in production. Map priors, occlusion-boundary risk scoring, and motion priors let production stacks (Waymo / Mobileye / Bosch) initiate conservative slow-downs ~6–8 m before LOS clears — enough for safety in this scene, but at the cost of nuisance braking on every blind corner. Our NLOS layer adds a further ~12 m of advance warning and a target-confirmed trigger (AND-gated on independent physics) so braking fires only when the occluded object is real. The +2.35 s margin is over the naïve LOS baseline; over the occlusion-aware baseline it is ~1.5 s plus a sharply lower false-brake rate. Both gaps are quantified in a pilot.

§4 · Sensor signatures · multipath + diffraction

Two independent physics, one M t signature.

The hidden child is invisible directly but produces a weak return in both sensors through different physical mechanisms. The 77 GHz mmWave radar picks it up via wall-bounce multipath (long wavelength diffracts around the lead parked car). The 40 kHz ultrasonic array picks it up via edge diffraction (short wavelength bends around the lead car's near edge). The fact that the two returns are independent physical phenomena is what lets the AND gate suppress noise without losing the real signal.

RF modal : m RF n (t) = |FFT n { flatten( F RF az,rng (t) ) }|,   n = 0 … 7 US modal : m US n (t) = |FFT n { F US az (t) }|,   n = 0 … 7 M t : [ m RF 0…7 (t) , m US 0…7 (t) ] ∈ ℝ 16 fused gate : ρ fused (t) = √( max(0, ρ RF (t) − τ RF ) · max(0, ρ US (t) − τ US ) )

§5 · Sealed perception history

Each M t is signed by an FPGA-resident PUF key, then chained.

One M t snapshot every 100 ms, HMAC-SHA256 signed against the previous transition hash with a key derived from a PUF inside an FPGA. An 8 % alteration to a single coefficient of any historical snapshot breaks every downstream signature — both the OEM and a regulator can independently verify offline.

Fig. — 10-snapshot perception chain. 8 % coefficient alteration to M 4 in storage: 4 / 10 links validate (clean baseline = 10 / 10). All H k for k ≥ 4 reject.

σ t = HMAC K PUF (H t−1 ‖ M t )

H t = SHA256(H t−1 ‖ M t ‖ σ t )

verify HMAC K PUF (H t−1 ‖ M t ) ≟ σ t   ∀ t

MAC primitive

HMAC-SHA256, 256-bit PUF-derived key

Key custody

AWS F2 FPGA (AFI agfi-085919c09c35982c3 ) in the lab today; production target = ASIL-rated automotive FPGA inside the perception ECU

EDR / data-recorder integration

Each sealed M t persisted in the existing event-data-recorder ring buffer; binary-compatible with current EDR R&R format

Tamper sensitivity (measured)

8 % change to a single modal coefficient → 100 % of downstream links reject

Multi-party verification

OEM, regulator (NHTSA / KATRI / UNECE), and accident investigator each hold the public verification key

Chain frame rate

10 Hz seal — comfortably within an automotive perception ECU's compute budget; SHA-256 + HMAC ≈ μs-class on a $5 FPGA

§6 · Cross-cutting claim XC1 — verified-prior recursive smoother

The chain doesn't just attest the past — it improves the present .

A perception loop that blends each new measurement with its cryptographically verified past is both more consistent (lower trigger-time variance) and self-defending : when the chain is tampered, the smoother weight on the prior collapses to zero and the receiver degrades to a memoryless single-shot detector — never below baseline. Both the gain and the collapse are measured here over 80 Monte-Carlo noise realizations.

M̂ t = w · M̂ t−1 + (1 − w) · sensor t   if chain verification at link t−1 succeeds M̂ t = sensor t   if chain verification at link t−1 fails (memoryless collapse) w = 0.7 (trust prior 70 %)

Receiver

Mean trigger distance (m to corner)

σ over 80 trials (m)

Safe rate

Single-shot (no prior)

19.1

1.45

100 %

Naive 5-frame moving average (no chain)

19.5

1.00

100 %

XC1 verified-prior (chain OK)

19.7

0.95

100 %

XC1 under chain tamper (self-collapse)

19.1

1.45

100 %

Key result: σ drops 35 % when the chain is intact (1.45 → 0.95 m). Under tamper, σ recovers exactly to single-shot (1.45 m) — never worse than baseline. This is the defining behaviour of XC1 — the chain integrity is the smoother's control input, making the perception layer and the security layer one closed loop.

§6.5 · HD-map M t -rail + AI dispatch · methodology

Mapped Tier-1 cities = Rail-applicable sub-scope.

The §6 verified-prior smoother is the kernel. This section is the deployment surround : on a mapped Tier-1 city (SF Bay, NYC, central London, Tokyo) the environment (lane geometry, building polygons, RSU positions, static obstacles) can be pre-laid as a rail. An AI dispatcher then chooses the right per-frame pipeline based on what the rail predicts about LOS / occlusion / weather / map-staleness. This section is methodology only — there is no fleet-deployment data — but it lays out (A) published challenges, (B) Tier-1 / production state of the art (Mobileye REM, HERE HD Live Map, Waymo Driver, Apollo, V2X cooperative perception), and (C) where the Axowl Rail + AI dispatch + chain primitive plugs in.

A. Why urban NLOS perception is hard

Challenge

Mechanism

Published source

NLOS pedestrian darting-out

Child / cyclist hidden behind parked vehicle or T-junction wall — appears in the ego-vehicle FOV with sub-second time-to-collision; conventional sensors cannot see it before egress

arXiv 2508.04033 (Aug 2025) — Radar-based NLoS pedestrian localization for darting-out near parked vehicles; arXiv 2508.02348 — mmWave radar NLoS at T-junctions

Multipath ghost targets

Urban building reflections create false radar returns ("ghost") that degrade detection and waste compute; same mechanism that enables NLOS detection also creates false-alarms if not filtered

arXiv 2503.05413 — Hybrid approach for extending automotive radar to NLOS urban scenarios; IEEE AESS 2024

HD-map staleness

Construction / lane closure / road repair makes the map diverge from reality; the Waymo CA DMV 2024 report cited 3 disengagement events caused by map-update delay

California DMV Autonomous Vehicle Disengagement Report 2024 (Waymo); arXiv 2509.12632 — Maps for AD full-process survey 2025

Adverse weather degradation

LiDAR scattering in heavy snow / dust / fog; camera glare at low-sun angle; degraded performance in 5-10 % of operational time

Waymo Driver heavy-rain recall 95.2 % (2024 report); standard CARLA / nuScenes weather benchmarks

B. Current Tier-1 / production state of the art

System

Function

Reference

Mobileye REM™ (Roadbook)

Crowd-sourced HD-map from 40M+ vehicles globally (as of 2020); ~10 kb/km bandwidth; continuous cloud-updated map of lanes, signs, drivable area. Partnerships with BMW, GM, VW, HERE.

Mobileye REM (mobileye.com/technology/rem); HERE + Mobileye joint mapping

Waymo Driver

Target-level + feature-level fusion (cross-modal attention on LiDAR voxel + image); 95.2 % heavy-rain recall; 250 ms end-to-end latency. HD-map dependent.

Waymo AI & robotics deep-dive 2024 (techrxiv); CA DMV reports

Baidu Apollo HD Map

Open-platform HD-map + perception stack; lane-level localization + cross-modal sensor fusion

apollo.auto (open-source platform)

Cooperative Perception (V2X)

V2V / V2I / V2N message exchange of sensor features and detection results; relay selection + store-carry-forward sustain NLOS coverage in obstructed corridors

arXiv 2310.03525 — V2X Cooperative Perception Survey 2023; arXiv 2307.06615 — "NLOS Dies Twice"; arXiv 2512.00490 — Cooperative Safety Intelligence Survey

Hybrid radar NLOS (CNN + multipath physics)

CNN-based reflective-surface detection + physical electromagnetic propagation model — jointly estimates surface and target without auxiliary sensors or Doppler

arXiv 2503.05413 (March 2025)

C. Axowl Rail + AI dispatch + chain — methodology (where the engine plugs in)

Layer

What plugs in

Where it would help

Pre-laid road rail

HD-map prior (Mobileye REM / HERE / Apollo / custom) — lane geometry, building polygons, RSU positions, parking-zone static obstacles — combined with the §3 phase-invariant modal-decomposition forward model

Each frame starts from a known road skeleton; perception only refines the dynamic objects on top

V2X cooperative fusion

RSU + neighbouring-vehicle sensor features (per arXiv 2310.03525 + 2307.06615 NLOS bypass), authenticated through the §5 PUF chain so the ego vehicle can trust a remote sensor measurement

NLOS pedestrian detection extends beyond ego FOV; V2X-fusion is only safe if every remote message is chain-verified (else V2X becomes a spoof vector)

AI dispatch trigger

LOS lost / occlusion detected → NLOS modal pipeline (radar multipath, §3 – §4) · map staleness > 7 days for a corridor → trigger REM-style online crowd-update · adverse-weather (snow / dust / fog) detected → degraded chain-only mode with elevated alert

Same architectural pattern as Space §S1.7 (regime classifier + per-regime pipeline + explicit out-of-scope alarm) and Defense §4.8 (WOD / pad-motion triggers)

Chain-anchored integrity

Every camera / LiDAR / radar / V2X frame is PUF-chain authenticated at the natural per-frame cycle (camera 30 Hz, LiDAR 10 – 20 Hz, radar 50 – 100 Hz); spoofed sensor input → chain fail → smoother weight collapses to single-shot (§6 XC1 behaviour)

Per-frame integrity = real-time safety. Sealed EDR record = post-incident accident-investigation primitive (§7 / §8)

Sub-scope — where this Rail applies. ✅ Mapped Tier-1 cities with HD-map coverage and recent REM-style crowd updates (SF Bay, NYC, central London, SF Bay, NYC, Tokyo, Shanghai, Munich). 🟡 Lite-map deployments (2021+ industry trend toward lightweight maps): partial rail — coarse geometry yes, fine landmark prior no. ❌ Rural / unmapped roads, off-road, active construction zones, heavy-snow / dust whiteouts — Rail is not applicable; chain primitive carries over as a tamper-evident EDR but the smoother falls back to §6 single-shot behaviour. Methodology status (no fleet-deployment data). Validation requires (i) integration with an actual HD-map provider's SDK (Mobileye REM, HERE, TomTom, or in-house OEM map), (ii) closed-track or simulation NLOS scenarios over both clean and stale maps, and (iii) V2X stack integration (C-V2X PC5 or DSRC) with a real RSU partner. The current claim is " the Axowl Rail + AI + chain primitive plugs into the urban-NLOS perception problem in these four places " — not a CEP or false-alarm number for a specific OEM stack.

§7 · Cross-cutting claim XC9 — fault tolerance under natural physical degradation

1 M vehicles × 10 years × natural silicon aging. σ does not budge.

Adversarial tamper of a PUF is statistically negligible — the PUF is intrinsically tamper-evident, and XC2 already covers detection. The real production failure modes over an ASIL-D 10-year automotive lifetime are physical : NBTI / HCI / TDDB silicon aging, cosmic-ray SEU events, vibration- and temperature-induced response drift, connector corrosion, power-rail glitches. XC9 is the patent's answer — N = 5 independently- fabricated PUFs, K = 3 majority quorum, dynamic quarantine on consecutive-miss persistence, periodic re-probe of quarantined PUFs (transient SEU recovers automatically), last-known-good rollback through the rare quorum-miss frames. Across 200-trial Monte Carlo of four production-realistic failure scenarios — including two simultaneously failing PUFs — σ of trigger distance is statistically indistinguishable from a clean 5-PUF baseline. "Avoiding a rubbish truck only to crash into a sewage truck" — that scenario does not happen here.

Scenario

Failure mode

End-of-run mean N eff

σ trigger distance (m)

S1 clean

None — all 5 PUFs healthy (yr 0)

5 of 5

1.00

S2 transient SEU

Cosmic-ray strike on PUF #0, frames 25–35, recovers

5 of 5 (recovered)

1.00

S3 persistent fail

PUF #0 silicon defect (NBTI end-of-life)

4 of 5

1.00

S4 worst case

PUF #0 hard fail + PUF #1 thermal-cluster fail at frame 40

3 of 5

1.00

(reference) single-shot

No prior, no chain, no smoother

1.45

Key result: σ identical (1.00 m) across all four production-lifetime scenarios — including the worst case where 2 of 5 PUFs are permanently quarantined and the verifier is operating on a 3-of-3-of-5 minority quorum. Compare to the single-shot baseline 1.45 m. Fault tolerance is 31 % σ-reduction-preserved regardless of which physical failure occurs . The verified-prior smoother continues to operate via XC9a last-known-good rollback through every quorum-miss frame — perception never falls back to single-shot operation, eliminating the "rubbish-truck / sewage-truck" failure mode.

N = 5   (odd, independent silicon wafer lots) K = ⌈N eff / 2⌉ + 1   (default majority, recomputed as N eff drops) quarantine : 5 consecutive miss frames per PUF re-probe : every 30 frames (≈ 3 s at 10 Hz) — restores transient failures service flag: cumulative quarantine ≥ 1 hr → OEM telemetry alert

§8 · Cross-cutting claim XC4 — multi-party offline verification

OEM signs once. Three independent parties verify offline. Tamper is detected identically by all.

The PUF signing key resides in the OEM's perception ECU. The corresponding public verification key is distributed to N independent parties before chain production (OEM Tier-1, NHTSA / regulator, insurer / liability). Each verifies the chain offline using only the public key and the sealed records — no communication with the OEM required . Under tamper, all three independently detect the same break, at the same link. The OEM cannot unilaterally rewrite history.

Verifier

Holds

Clean chain

OEM tampers M 4 (8 % coeff)

OEM (Tier-1, key custodian)

PUF key + public key

10 / 10 PASS

4 / 10 PASS → fails at M 4

NHTSA / regulator

public key only

10 / 10 PASS

4 / 10 PASS → fails at M 4

Insurer / liability

public key only

10 / 10 PASS

4 / 10 PASS → fails at M 4

Key result: all three verifiers arrive at the identical first-fail link (M 4 ) without communicating. The OEM cannot construct a tampered chain that some verifiers accept and others reject. This is the cryptographic basis on which regulator / underwriter / accident-investigator independence rests.

§9 · How this stacks up against current AV/ADAS stacks

A layer no other vendor has — bolts on top of what they already ship.

Our pipeline is sensor-stack-agnostic. It does not replace LiDAR, cameras, or radar — it extracts information out of existing returns that no current fusion stack uses, and seals the result.

Vendor / stack

What they have today

What our IP adds

Tesla (vision-only)

Eight cameras

Radar return + acoustic NLOS + sealed perception chain

Waymo / Cruise (full stack)

LiDAR + camera + radar

NLOS layer (radar-acoustic fusion) on top of existing perception

Vayyar (RF-only NLOS)

mmWave-only NLOS

Acoustic-fusion modal extraction + chain seal

Mobileye

Camera + radar + REM

NLOS layer + cryptographically sealed REM

Bosch / Continental / ZF

Production radar + ultrasonic Tier-1 stack

Our modal-fusion algorithm + chain = production-ready

The defensible position is the three primitives simultaneously : multi-modal RF + acoustic fusion (claims 4 and 12), Chladni-style modal decomposition (claim 1), and FPGA-PUF sealed chain (claim 1). No incumbent has any single one of these; the three together are independently filed.

§10 · Integration paths

Three deployment shapes.

In-vehicle perception ECU

Onboard FPGA / SoC module that consumes the existing radar and ultrasonic raw streams. Tight integration with the AEB / ACC chain. ASIL-D path, longer cycle.

Fleet-telemetry NLOS service

Cloud-side fusion service that consumes uploaded raw radar / ultrasonic frames from OEM telematics. No ASIL impact; suitable for retrofit and post-incident analysis.

EDR-log signer

Sealed M t records produced alongside existing event-data-recorder data. The chain integrates with whichever perception stack the OEM ships (or none — RF-only fallback works).

Standards alignment

ISO 26262 (functional safety), ISO 21434 (cybersecurity), UNECE R155 / R156 (CSMS / SUMS), NHTSA FMVSS 150 (EDR), UN GTR on AEBS (pedestrian)

Licensing model

Per-vehicle IP block, per-fleet software license, or per-mile NLOS-as-a-service — negotiated via the lead financial advisor

§11 · Operational scenarios

Where the rail pays back.

Urban AEB — child / cyclist NLOS

The cohort that drives most EU NCAP / IIHS demerit points today: pedestrian-from-occlusion. RF + ultrasonic fusion is the only commodity-cost answer.

Commercial fleets / urban delivery

Last-mile delivery vans operate where pedestrian NLOS is the dominant accident class. The chain doubles as insurance-grade incident evidence.

L2+ / L3 autonomy

"Hand-back" disengagement events need a defensible record of what the car could and could not perceive. The chain makes this audit-grade.

Defense — convoy / urban patrol

Tactical vehicles need NLOS detection of personnel and IEDs around corners. Sealed chain prevents tamper of patrol logs.

Insurance / liability underwriting

Insurers can price policies based on chain-verified perception coverage rather than self-reported ADAS-feature lists.

Recall / defect investigation

NHTSA / KATRI / regulator investigations get a tamper-evident perception log without trusting the OEM's logging stack — closes the gap that today drives multi-year defect probes.

V2X / cooperative perception

Once sealed, M t snapshots are safe to broadcast to neighbouring vehicles — a tampered-with cooperative NLOS report is rejected at the receiver.

OEM software-update assurance

Each perception software update can be tied to a chain-anchored M t baseline — regression in NLOS performance becomes cryptographically detectable.

§12 · Honest limitations

What this page does not show.

2-D point mass. No suspension dynamics, no tire model, no pitch-on-braking, no production AEB controller. Vehicle dynamics matter for stopping-distance claims; production work needs CarMaker / VTD / Carla.

Synthetic sensor model — first real-data validation done (seq20, 2026-06-19). The RF and ultrasonic returns on this page's CPNCO demo are analytic stand-ins for full electromagnetic / acoustic simulation. On real data, the modal stack now runs end-to-end on K-Radar (KAIST, NeurIPS 2023 Datasets & Benchmarks) 4D-radar tensors, and on sequence 20 it detected 5 of 5 radar-invisible pedestrians a median +4.6 s before conventional radar (see the K-Radar section at the top of this page). This is one validated sequence with a clear NLOS window — not a full-dataset NLOS mAP across all 1,042 radar-invisible pedestrians / 988 frames, and not yet the Doppler 4D test. Tier-1-supplied raw radar tensors (Continental ARS548, Bosch MRR) and ultrasonic edge-bench data remain the production next step.

No clutter / no other road users. A real urban scene has dozens of multipath sources. Our AND-gated geometric mean is a deliberately conservative fusion designed to suppress them, but the false-alarm rate under realistic clutter is to be measured on raw OEM data.

Modal extractor is a deliberate simplification. Production work would use a wavelet basis or a learned-feature head rather than a fixed FFT. The chain primitive is independent of the extractor choice.

FPGA is software-emulated in this demo. The chain in §5 is computed in Python against the same PUF-key constant the real FPGA uses. A separate program of work has the same HMAC running on an AWS F2 FPGA AFI. An ASIL-D-rated automotive FPGA is the production target.

Chain content vs chain attestation. The chain attests which M t was produced when . It does not encrypt the modal state nor attest to driver or platform identity. Identity binding and confidentiality are separate primitives that stack on top.

No driver-in-the-loop study. The advance-warning number (+2.35 s) is the perception margin only. End-to-end avoidance also depends on the AEB controller and the driver-override budget.

What a perception evaluator will ask first — and where it is answered

Question

Answered here?

Where it gets answered

False-alarm rate per hour?

No

Pilot Phase A — fixed threshold on a held-out scenario library, FAR/h reported per ODD bucket

Does it generalise from child to adult / cyclist / animal?

No — child only

Pilot Phase B — labelled multi-target dataset, recall × class

Recall in the same scene over 100 runs?

Indirectly (§6, 80 MC trials, single scene)

Pilot Phase A — recall + variance across noise + clutter realisations

Headline false-brake rate on empty multipath-rich corners?

No

Pilot Phase A — empty-scene null-trial cohort with identical fusion threshold

Ultrasonic 25 m — real or simulator artefact?

Simulator ceiling (see §2 caveat)

Pilot Phase C — edge-bench measurement on OEM-supplied transducer array

Reproducible on a different radar vendor / different car?

No — single synthetic model

Pilot Phase D — re-run §3 against ARS548 / MRR raw tensors, Bosch / Continental / ZF

Effect of rain / snow / fog / wind?

No

Pilot Phase D environmental matrix

What collapses the fusion under tamper?

Yes (§6 XC1 — verified-prior smoother)

Already in §6 table — σ recovers to single-shot, never below baseline

What if 2 of 5 PUFs fail in the field?

Yes (§7 XC9 — σ unchanged at 3/5 quorum)

Already in §7 table

Can a regulator verify without trusting the OEM?

Yes (§8 XC4 — multi-party offline verify)

Already in §8 table

Reading this honestly: the integrity primitives (XC1 / XC9 / XC4 in §6 – §8) are answered on this page. The perception-accuracy questions (FAR/h, ROC, generalisation, cross-vendor) are not — they are the scope of the pilot. A licensee who needs (2) only — the sealed perception chain — does not need to wait for the pilot.

§13 · Patent claim map — what each section demonstrates

16 claims, one page, every one tied to a figure.

For IP review and underwriting: the table below maps each cross-cutting (XC) and automotive-specific (A) claim onto the page section and figure that demonstrates it. Cross-cutting claims appear independently in 4 – 5 other domain demos (telco, antenna, oil-gas, aviation, defense), making them generalizable rather than vehicle-specific.

Claim

One-line description

Demonstrated in

Also appears in

XC1

Verified-prior recursive smoother — chain integrity gates fusion weight

§6 · xc1_smoother.svg

antenna, telco, defense, oil-gas, aviation

XC2

Cascade tamper invalidation — single bit kills all downstream links

§5 · chain_diagram_drive.svg

all 5 other domains

XC3

Canonical timestamp-prefixed serialization (big-endian + float32)

script (line 304-310: canon() )

antenna, aviation

XC4

Multi-party offline public-key verification — 3 independent verifiers

§8 · xc4_multiparty.svg

telco, defense, oil-gas, aviation

XC5

Industry-standard format embedding — EDR ring buffer compatible

§5 spec-grid · "EDR / data-recorder integration"

SEG-Y, O-RAN, ETSI LI, FDR/QAR

XC6

Cloud TEE + HSM PUF emulation alternative

§10 spec-grid · "Fleet-telemetry NLOS service"

oil-gas, defense

XC7

FPGA hardware HMAC engine + metering counter

§5 spec-grid · "Key custody" / "Chain frame rate"

aviation, defense

XC8

Sidecar retrofit appliance — no PHY/ECU certification impact

§10 spec-grid · "Fleet-telemetry NLOS service"

telco, aviation

XC9 ★

Fault-tolerant odd-N (N=5, K=3) PUF quorum under natural physical degradation + LKG rollback + dynamic quarantine + re-probe

§7 · xc9_faulttolerance.svg

candidate for telco, defense, aviation, oil-gas (any safety-critical fleet)

A1

Multi-physics NLOS fusion gate (mmWave + ultrasonic AND)

§3 · anomaly_curves.svg + §4 · sensor_fields.svg

automotive-only

A2

AND-gated geometric-mean fusion with noise-floor subtraction

§4 math block

automotive-only

A3

Phase-invariant FFT magnitude modal signature

§4 math block · M t definition

automotive-only

A4

Real-time perception sealing at sensor frame rate (10 Hz)

§5 spec-grid · "Chain frame rate"

automotive-only

A5

OEM-installed commodity sensors only — no new hardware

§1 use card · "5 – 50× cheaper than LiDAR"

automotive-only

A6

EDR ring-buffer binary compatibility (NHTSA FMVSS 150)

§5 spec-grid · "EDR / data-recorder integration"

automotive-only

A7

V2X cooperative perception with chain-anchored trust

§11 use card · "V2X / cooperative perception"

automotive-only

Why this matters: a competitor that reads only the automotive page sees 7 reasons we win in this market. A patent examiner who reads the cross-cutting block sees that 8 of these patterns are independently demonstrated in 4 – 5 other domain demos — i.e., they are not automotive tricks, they are generalizable inventions.

§14 · For your perception team

Run it on your own raw radar / ultrasonic data.

The Python pipeline that produced every figure on this page is available under NDA. The recommended first step is to repeat §3 against OEM-specific raw radar tensors and edge-bench ultrasonic data — replace the analytic sensor model with a real-data raster and re-measure trigger distance and false-alarm rate against your in-house fusion. The chain primitive (§5) is sensor- independent and can be evaluated separately from the fusion work.

Request engineering briefing Read the whitepaper