ON-VEHICLE VOICE & DIAGNOSTIC ASSISTANT

The Business Problem

What the vehicle already knows

A modern passenger or commercial vehicle produces thousands of diagnostic events every journey. Powertrain, chassis, body and ADAS controllers exchange messages across CAN and CAN-FD, legacy comfort functions sit on LIN, and the emissions related subset is exposed through the OBD-II port.

Fault memory, freeze frame data, readiness monitors and live sensor values are all present. Almost none of that reaches the person driving. The driver gets a lamp on the cluster and, at best, a three word message. The interpretation happens later, at a workshop, using a diagnostic tool and a technician who was not there when the fault occurred.

Why cloud assistants do not solve it

The connected assistants deployed today push the question and often the vehicle context to a
backend. That introduces four problems at once.

Latency. A round trip over a mobile network adds hundreds of milliseconds before the model even
starts generating, and the variance is worse than the average.
Availability. Underground car parks, tunnels, rural roads, ferries and border crossings all break the
feature at exactly the moment a driver is most likely to need it.
Privacy and residency. Vehicle identification, location, driving behaviour and fault history are personal
data in most jurisdictions once they are linked to a person.
Recurring cost. Per query inference billed to the OEM for the life of the vehicle is a liability that grows
with fleet size and with feature success.

The cost of the status quo

Where the cost sits	What actually happens
Driver	Uncertainty about whether the vehicle is safe to keep driving, leading to either unnecessary alarm or ignored warnings.
Dealer network	Diagnostic time consumed on faults that were self explanatory, and no-fault-found visits that carry full labour cost.
Roadside assistance	Callouts for conditions that did not require a callout.
Warranty	Late intervention on faults that degrade into more expensive failures.
Fleet operator	Vehicles taken off route on precaution rather than on evidence.
OEM brand	The warning lamp remains the least understood interface in the car.

The requirement in one sentence

Explain vehicle state to the driver, in natural language, at the moment the fault appears, grounded in the manufacturer's own documentation, with no dependency on a network and no data leaving the vehicle.

Why now

Two things changed. Small language models in the three to four billion parameter range became genuinely capable at instruction following and summarisation, and quantization matured to the point where those models fit the memory budget of an automotive grade system on module. The reasoning layer can finally sit next to the bus it is reasoning about.

Solution Overview

The assistant is a single edge node on the vehicle network. It listens, reads live vehicle state, retrieves the relevant manufacturer documentation from local storage, reasons over all three, and speaks the answer back. Every stage runs on the AMD Kria KR260

Layered architecture

Layer 6 Interaction

Voice in, voice out

Beamformed microphone array, wake word, text to speech through the cabin audio path.

Layer 5 Reasoning

4-bit quantized language model

Instruction tuned small model, constrained to answer only from retrieved and observed facts.

Layer 4 Knowledge

Local retrieval index

Service manuals, DTC reference, repair procedures, vehicle specific history.

Layer 3 Vehicle state

CAN and OBD-II decode

DBC driven signal decode, UDS fault memory read, freeze frame capture.

Layer 2 Acceleration

Programmable fabric and DPU

Dequantization, matrix operations and audio pre-processing offloaded from the CPU cores.

Layer 1 Platform

Kria K26 SoM on KR260 carrier

Zynq UltraScale+ MPSoC, on-module DDR4, local NVMe or eMMC.

Design principles

Principle	How it is enforced
Read only on the bus	The CAN transceiver is configured listen only. The assistant physically cannot transmit a frame onto the vehicle network.
Grounded answers only	The model answers from retrieved document text and decoded signals. If neither supports an answer it says so rather than generating one.
No egress	There is no outbound data path in the reference design. Audio, faults and location never leave the node.
Deterministic core untouched	All safety relevant behaviour stays in existing ECU software. The assistant is an explanation layer, not a control function.
Bounded memory	Every component has a fixed memory allocation. There is no dynamic growth path that can exhaust the module under load.
Signed everything	Model weights, retrieval index and application image are all signature verified before they are allowed to load.

What this is not

This is not a voice control system and it does not command any vehicle function. It reads, explains and recommends. That boundary is what keeps the homologation and functional safety argument simple.

Hardware Architecture

Compute platform

The AMD Kria KR260 robotics starter kit carries the K26 system on module. The KR260 is chosen over the KV260 because the carrier already exposes the Ethernet, PMOD and general interfaces that vehicle and industrial integration needs, rather than a vision oriented interface set.

Element	Configuration
System on module	Kria K26, Zynq UltraScale+ MPSoC
Application processors	Quad core Arm Cortex-A53
Real time processors	Dual core Arm Cortex-R5F
Programmable logic	On-module FPGA fabric with BRAM and URAM
On-module memory	4 GB DDR4 with ECC support
Local storage	NVMe or eMMC, sized for model plus retrieval corpus
Boot	Secure boot chain with on-module non volatile storage

Vehicle side peripherals

CAN / CAN-FD transceiver (listen only) OBD-II J1962 harness LIN transceiver (optional)
Automotive Ethernet (optional)

Operator station

Four element MEMS microphone array Hardware acoustic echo cancellation
Class-D amplifier or head unit audio path Push to talk input (optional)

Power and thermal

The node is designed to run from switched vehicle 12 V through an automotive grade DC-DC stage with load dump and reverse polarity protection. Thermal design is the constraint that usually decides real world throughput, so it is treated as a first class requirement rather than a packaging afterthought.

Sustained rather than peak performance is the design point. The assistant must hold its token rate through a full duty cycle at cabin ambient, not just for the first thirty seconds on a bench.
Conducted cooling to the enclosure is preferred over fans in a cabin installation, for noise and for long term reliability.
Thermal throttling behaviour is characterised and the model scheduler is made aware of it, so degradation is graceful rather than abrupt.
Cold start behaviour at low ambient is validated separately, since DDR and storage timing both shift

Integration decision to take early

Whether the node is a standalone aftermarket module on the OBD-II port or an integrated function on an existing domain controller changes the harness, the thermal envelope and the homologation path. This decision should be taken before detailed design, not after.

Vehicle Interface and Signal Decode

Protocol coverage

Interface	Purpose in this architecture
CAN 2.0B and CAN-FD	Live signal observation across powertrain, chassis and body networks at production bus rates.
ISO 15765-2	Transport layer for segmented diagnostic messages.
UDS, ISO 14229	Fault memory read, freeze frame retrieval and data identifier reads where the OEM permits them.
OBD-II	Emissions related fault codes, live data, readiness monitors and the standardised subset available on every vehicle.
LIN	Comfort and body subsystems where the fault is relevant to the driver.
DBC or ARXML	Signal database that turns raw frames into named, scaled, unit-bearing values.

From frame to fact

Raw CAN traffic is useless to a language model. The decode stage converts it into a compact, typed vehicle state object that the reasoning layer can be given directly, with names and units the documentation also uses.

Vehicle state object

Why the snapshot reference is in the plan

Recording which world model snapshot the plan was built against makes the whole decision reproducible after the fact. When something goes wrong on a floor, the first question is always what the system believed at the time, and this answers it exactly.

Rate and buffering

High rate signals are decimated and windowed at the decode stage, not at the model.
A rolling buffer retains the seconds before a fault so freeze frame context is available even when the driver asks
a minute later.
Bus load is unaffected because the node never transmits.

Speech Front End and Response Path

Capture

Cabin audio is a harder problem than the model. Road noise, HVAC, wind, music and passengers all sit in the same band as speech, and the microphone is rarely where you would want it.

Microphone array capture at 16 kHz

Multi element MEMS array positioned for the driver seat.

Beamforming and acoustic echo cancellation

Steered toward the driver, with the vehicle's own audio output subtracted.

Noise suppression and voice activity detection

Runs on the fabric so the application cores stay free for inference.

Wake word or push to talk gate

Nothing is transcribed or retained until the gate opens.

Whisper Tiny or DistilWhisper transcription

Quantized to int8, running on device, audio discarded immediately after.

Response

The generated answer is spoken back through Piper TTS or Coqui TTS, streamed as it is produced rather than synthesised after the full answer is complete. Streaming matters more than raw token rate for how fast the system feels, because the driver hears the first words while the rest is still generating.

Stage	Design target
Wake to transcript	Complete before the driver finishes the sentence.
Vehicle state assembly	Read from the rolling buffer, no bus query needed.
Retrieval	Local index lookup, no network.
First spoken syllable	Sub second from end of utterance.
Full answer	Streamed continuously, no audible gap.

Status of figures

These are design targets for the reference configuration. They are confirmed on the customer hardware, cabin acoustics and duty cycle during the engineering phase, and the validated numbers replace them in the delivered specification.

Privacy posture

Audio is never written to persistent storage.
Transcripts exist only for the duration of the interaction unless the operator explicitly enables a diagnostic log,which is off by default.
There is no outbound network path in the reference design, so there is nothing to intercept.

Inference Stack and Memory Budget

Why four bit

A three billion parameter model at sixteen bit precision needs roughly six gigabytes for weights alone. The module has four. At four bit the same model comes down to approximately 1.7 gigabytes, which leaves working room for the key value cache, the speech models, the retrieval index and the operating
system.

The second effect matters more. Generating each token requires streaming the entire weight set out of DDR, so throughput is bound by memory bandwidth rather than by arithmetic. Halving the bytes per weight roughly doubles the achievable token rate.

Quantization approach

Planner responsibility	What it checks
Dependency ordering	Precedence expressed in the instruction is turned into an acyclic task graph and validated.
Feasibility	Is there a robot of the right type, free within the window, able to reach both locations.
Battery and duty	Does the assigned robot have enough charge for the full task chain plus a return margin.
Zone and access rules	Are all traversals and all destinations permitted for this robot type at this time.
Capacity	Does the destination have space, and is the load within payload limits.
Conflict	Does this plan contend with an already dispatched plan for the same robot, load or location.

Indicative memory budget

Choice	Rationale
Group-wise 4-bit	Per-group scales preserve accuracy far better than a single tensor-wide scale, at negligible memory cost.
GPTQ or AWQ	Calibration on representative automotive text keeps the quality loss concentrated away from the domain that matters.
Sensitive layers kept wider	Embeddings and the output head carry disproportionate error, so they are held at higher precision.
Domain evaluation set	Accuracy is measured on decoded fault scenarios and manual questions, not on generic benchmarks.

Arbitration stays with the fleet manager

The planner proposes. The existing fleet manager remains the single authority that dispatches, sequences against other traffic and resolves contention. Introducing a second scheduler would be the fastest way to create deadlock on a live floor, so the architecture deliberately avoids it.

Context discipline

Context length is a memory decision, not a convenience. The key value cache grows linearly with context and competes directly with the weights for bandwidth. The design uses a short working context, aggressive retrieval rather than long prompts, and a fixed cap that cannot be exceeded at runtime.

The DDR Pipeline

This is where the engineering effort actually goes. Theoretical bandwidth on the module is a ceiling nobody reaches, and the gap between theoretical and achieved bandwidth is the single largest determinant of how the system performs in a vehicle

The constraint

Every generated token requires the full weight set to cross the memory bus once. The upper bound on token rate is therefore achievable bandwidth divided by model bytes, and no amount of compute
optimisation moves it. Everything below is aimed at raising the achievable fraction.

Optimisation techniques applied

Tiling

Weight matrices tiled to on-chip memory

Tile geometry chosen against available BRAM and URAM rather than against the model shape.

Double buffering

Transfer overlapped with compute

One tile is processed while the next streams in over AXI, so the accelerator never waits.

Prefetch

Next layer fetched during current layer

Layer boundaries are the most common stall point and are eliminated deliberately.

On-fabric dequant

Four bit bytes cross the bus

Dequantization happens after transfer, not before, so the bus carries the compressed form.

Burst alignment

Access patterns matched to DDR pages

Row activation and refresh overhead is minimised by laying weights out for sequential access.

What goes wrong when this is skipped

Symptom	Usual root cause
Token rate far below estimate	Dequantization on the processor side, so sixteen bit values are crossing the bus.
Throughput collapses with context	Key value cache traffic contending with weight streaming for the same bandwidth.
High variance between runs	Unaligned access patterns causing DDR row thrash.
Accelerator utilisation low	No double buffering, so compute idles during every transfer.
Good bench, poor vehicle	Thermal throttling not characterised, so sustained performance was never measured.

Fabric offload beyond memory movement

The programmable logic is also used for the audio front end, for the vision path where one is present, and for matrix operations that map cleanly to the DPU. The objective is to keep the application cores free for orchestration and decode, rather than to have them competing with the model for the same resources.

Measurement discipline

Achieved bandwidth is measured directly, not inferred from token rate.
Sustained performance is reported over a full duty cycle, never as a peak figure.
Every quoted number carries the model, quantization scheme, context length and ambient temperature it was
obtained at.

Knowledge Layer and Grounding

Corpus

The assistant answers from the manufacturer’s own material, held on local storage and versioned against the vehicle build. Nothing is answered from model parameters alone.

Service and workshop manuals Diagnostic trouble code reference Repair procedures and labour guides
Owner handbook Technical service bulletins Vehicle specific repair history

Ingestion pipeline

Layout aware document parsing

Tables, torque specifications and procedure steps preserved as structure, not flattened text.

Section anchored chunking

Chunks follow document sections so a citation points somewhere a technician can find.

Embedding and index build

Compact sentence embeddings, index built offline at release time rather than on the vehicle.

Hybrid retrieval

Dense similarity combined with keyword matching, because fault codes are exact strings.

Grounding policy

The reasoning layer operates under a strict contract. It may use the decoded vehicle state and the retrieved document text. It may not use anything else.

Situation	Required behaviour
Retrieval returns strong match	Answer, and internally record which section was used.
Retrieval is weak or empty	State that the vehicle documentation does not cover the question and recommend contacting service.
Question outside scope	Decline and redirect, rather than improvising.
Fault is safety relevant	Give the manufacturer's stated instruction verbatim in substance, and do not soften it.
Conflicting sources	Prefer the newest technical service bulletin and say that guidance was updated.

Version pinning

The retrieval corpus is pinned to the vehicle build. A manual that describes a different variant is worse than no manual, so corpus version, model version and application version are released and updated as one signed bundle.

Evaluation

Answer quality is measured against a fixed regression set of decoded fault scenarios with known correct guidance, refreshed whenever the corpus or the model changes. Retrieval accuracy and answer faithfulness are scored separately, because they fail for different reasons and need different fixes.

Worked Example, End to End

Driver

“My engine light is on.”

Step 1 Capture

Utterance transcribed on device

Beamformed, echo cancelled, transcribed by Whisper Tiny. Audio discarded.

Step 2 Vehicle state

P0420 confirmed, bank 1

Read from the rolling buffer with freeze frame at the moment of detection.

Step 3 Retrieval

Three grounded sources returned

DTC reference entry, catalytic converter section of the service manual, and two prior occurrences from this vehicle's own history.

Step 4 Reasoning

Severity action derived

Catalyst efficiency below threshold, emissions related, not immobilising, service recommended within a bounded distance.

Step 5 Response

Streamed to the cabin speaker

Speech synthesis begins before generation completes.

Assistant

“Your vehicle has detected reduced catalytic converter efficiency. Driving is safe for short distances, but service is recommended within 150 kilometres. This is the third time this fault has appeared, so it is worth mentioning at your next visit.”

Follow up handling

Driver

“Can I still drive to Chennai tomorrow?”

Assistant

“That distance is beyond the recommended service interval for this fault. The vehicle will drive, but emissions performance is degraded and continued driving may affect other components. I would have it checked before a long journey.”

What made the second answer possible

Conversational context is retained locally for the duration of the session. The assistant did not re-read the bus or re-run retrieval. It reasoned over state it already held, which is why the follow up is faster than the first answer.

Safety, Security and Homologation

Functional safety position

The assistant is deliberately kept outside the safety path. It observes and explains. It does not actuate, it does not command, and it does not suppress or modify any existing warning

Aspect	Position
Bus access	Listen only at the transceiver, enforced in hardware.
Existing warnings	Unchanged. The cluster behaves exactly as it does today.
Classification	Treated as a quality managed item, outside the ASIL decomposition, subject to confirmation with the OEM safety team.
Failure mode	Silence. If the assistant fails, the driver is exactly where they would have been without it.
Driver distraction	Voice first interaction, assessed against the applicable distraction guidelines for the target market.

Cybersecurity

Even with no outbound path, the node is an attached device on a vehicle network and is treated as part of the attack surface.

Secure boot from the on-module root of trust, with signature verification at every stage.
Model weights and retrieval index signed and verified before load, so a tampered corpus cannot change what the assistant tells the driver.
Update path aligned to UNECE R156 software update management expectations.
Threat analysis and risk assessment aligned to UNECE R155 and ISO/SAE 21434, carried out against the specific vehicle architecture.
No inbound remote interface in the reference design. Servicing is physical or through the OEM’s existing authenticated update channel.

Long term consideration

A vehicle stays on the road for fifteen years or more. Signatures applied today have to remain trustworthy for that whole period, which is why hash based or lattice based post quantum signing of model and corpus artefacts is worth designing in now rather than retrofitting later.

Deployment and Lifecycle

Release bundle

Model, retrieval corpus and application are released together as one signed bundle with a single version number. Mixing versions is the most common way an on-device assistant starts giving wrong answers, so the design removes the possibility.

Build

Quantize, index, evaluate, sign

Regression set must pass before a bundle is eligible for release.

Distribute

Existing OEM update channel

No new backend is introduced. The bundle rides what the OEM already operates.

Install

A/B partition with verification

The new bundle is verified before it becomes active.

Rollback

Automatic on failed health check

The previous known good bundle is retained until the new one has proven itself.

Fleet observability without data egress

Quality has to be monitored without shipping conversations off the vehicle. The design uses aggregate, non-identifying counters read at service, such as refusal rate, retrieval miss rate and interaction counts by fault family. These indicate where the corpus is thin without exposing what any driver said.

Delievery phases

Phase	Scope	Exit criteria
Phase 0 Assessment	Vehicle architecture review, bus access and DBC availability, corpus audit, target variant selection.	Agreed reference configuration and integration decision.
Phase 1 Bench prototype	Model quantized and characterised on KR260, decode stack against recorded bus traces, retrieval built on real manuals.	Measured token rate, memory footprint and thermal envelope on the bench.
Phase 2 In-vehicle pilot	Installed in target vehicles, cabin acoustics tuned, duty cycle and thermal behaviour validated in use.	Sustained performance and answer quality against the regression set in the vehicle.
Phase 3 Productionisation	Security case, update path, homologation support, manufacturing and service documentation.	Release candidate bundle and signed off security assessment.
Phase 4 Fleet rollout	Staged deployment with rollback, corpus expansion by variant and market.	Agreed field quality thresholds held over the rollout window.

Benefits and Measurement

Benefits realised

Offline operation

Full function in tunnels, underground parking, remote roads and any market without reliable coverage.

Sub second response

No cloud round trip anywhere in the critical path, and no variance from network conditions.

No customer data leakage

Audio, faults, location and driving behaviour never leave the vehicle boundary.

Real time diagnostics

Explanation delivered at the moment the fault appears, with freeze frame context intact.

Lower support load

Fewer avoidable dealer visits and roadside callouts for self explanatory conditions.

Predictable cost

Inference cost is capital, not a per query operating charge that scales with fleet size.

Regulatory headroom

On device processing simplifies the data residency and privacy position in every market.

Brand differentiation

The warning lamp stops being the least understood interface in the car.

KPI framework

These are the measures worth instrumenting from the pilot onward, because they are the ones a sponsor will be asked about.

Measure	What it tells you
Containment rate	Share of driver questions answered without a service contact.
Time to explanation	Interval between fault detection and the driver understanding it.
Refusal rate	How often the corpus does not cover the question, which is the main signal for where to expand it.
Retrieval miss rate	Whether the index or the corpus is the weak link.
No-fault-found visits	Direct dealer cost avoided.
Roadside callout rate	Direct assistance cost avoided.
Sustained token rate	Whether the thermal design is holding in the field.

Baseline first

Every one of these needs a pre-deployment baseline. Without it the pilot will produce numbers that cannot be defended, which is the most common reason a technically successful edge pilot fails to get funded into production.

Engineering Considerations and Risks

Known hard problems

Risk	Mitigation
Thermal ceiling in cabin ambient	Characterise sustained rather than peak performance early, and design the enclosure for conducted cooling before software optimisation begins.
Corpus and build drift	Single signed bundle with one version number covering model, corpus and application.
Ungrounded generation	Strict retrieval contract, explicit refusal path, and a regression set that specifically tests refusal behaviour.
Bus access restrictions	Confirm DBC or ARXML availability and permitted UDS services with the OEM in Phase 0, before any design commitment.
Cabin acoustics	Microphone placement and beamforming tuned per vehicle model, not carried over from a reference cabin.
Multi language markets	Speech recognition, generation and corpus all have to move together, and each language multiplies validation effort.
Long lived signatures	Plan the migration to post quantum signing of artefacts against the service life of the vehicle.

Open questions for the customer

Which fleet manager is in use, and what interface does it expose for plan submission and state
subscription?
Does a warehouse management system integration exist that can supply load identifiers, or is the world
model limited to fleet state only?
Are instructions issued from fixed stations, roaming headsets, or both?
What is the authorisation model? Can any operator issue any instruction, or is it role based?
Which languages and how many sites are in scope for the first release?
Who is the safety authority for the installation, and what evidence will they want that the language
layer cannot reach a safety function?

Recommended first step

A short assessment engagement against one vehicle variant, producing a measured performance envelope on the KR260 and a corpus readiness report. That converts the design targets in this document into numbers you can build a business case on, before committing to a full programme.