Batchalign

Welcome to Batchalign. Batchalign is a audio and ML pipeline for producing and enriching CHAT transcripts. It supports transcription, forced alignment, morphotagging, utterance segmentation, translation, diarization, comparison, and the Batchalign desktop shell.

User Guides

User guides for getting started are available via the following links:

• basic usage
• cleanup

Installation

Batchalign is available in two variants: as an application or as an a command-line tool. Downloads for both versions follows:

GUI Application

Press the blue rectangle corresponding to the operating system which you use to download the Batchalign application:

Finding the latest installers…

All releases and other builds

Terminal Interface (Advanced)

For the advanced terminal interface, the following scripts allows you to install Batchalign via uv.

On macOS or Linux:

curl -LsSf https://raw.githubusercontent.com/TalkBank/batchalign/main/bootstrap/bootstrap.sh | sh

On Windows, from PowerShell:

powershell -NoProfile -ExecutionPolicy Bypass -Command "Invoke-Expression ((Invoke-WebRequest -UseBasicParsing 'https://raw.githubusercontent.com/TalkBank/batchalign/main/bootstrap/bootstrap.ps1').Content)"

Batchalign documentation

Use Batchalign to transcribe recordings, align checked transcripts with audio, and add language analysis to CHAT files.

If you are new to Batchalign, begin with the Quick Start. It walks you through opening a command window and processing your first files.

For common clinical and research workflows, the Batchalign guide helps you choose the next task. The sidebar also includes supporting CHAT-format, Chatter, CLAN, and editor material, grouped by the same kinds of help.

Batchalign documentation

Batchalign helps you turn conversation recordings into CHAT transcripts and add language analysis to transcripts you have checked.

Start here

If this is your first time using Batchalign, follow the Quick Start. It shows you how to open a command window, check your installation, and process a transcript. You do not need programming experience.

Choose what you need

The documentation separates tutorials, how-to guides, reference, and explanation. Start with the kind of help you need; you do not need to read these pages in order.

Batchalign: tutorials

Learn by completing a guided task.

• Your first Batchalign run

Quick Start

Status: Current Last updated: 2026-09-13

Install Batchalign with the command in the TalkBank/batchalign README.

1. Open a command window

• macOS: Open Terminal. Apple explains where to find it.
• Windows: Open the Start menu, type PowerShell, and select PowerShell. Microsoft’s PowerShell guide includes startup help.
• Linux: Open the Terminal application from your applications menu. On many systems, Ctrl-Alt-T opens it directly.

2. Check the installation

Paste this command and press Enter:

batchalign --help

If Batchalign prints a list of commands, it is ready.

3. Process a file or folder

This example adds %mor and %gra tiers to every CHAT file in a folder:

batchalign morphotag "/Users/me/Documents/My Corpus"

In PowerShell, a Windows path looks like this:

batchalign morphotag "C:\Users\me\Documents\My Corpus"

Batchalign accepts either one CHAT file or a folder. Use quotation marks around paths containing spaces. Unless you specify an output folder, morphotag updates the files in place.

Before you start

Model downloads: The first use of a backend may download its models. Download size and startup time depend on the backend and language. Later runs reuse model files from disk.

Caching: Batchalign saves text and audio backend results in a local LMDB cache. Repeat runs can reuse matching results, although startup and file processing still take time. See Caching for details.

Basic command shape

batchalign [GLOBAL OPTIONS] COMMAND [COMMAND OPTIONS] [PATHS...]

• Global options go before the command.
• Most processing commands use -o/--out for a destination directory.
• Omitting -o/--out means in-place processing when the command supports it.

Transcribe audio to CHAT

batchalign transcribe ~/recordings/ -o ~/transcripts/ --lang eng

To use the local OpenAI Whisper package instead of the default Rev.AI engine:

batchalign transcribe ~/recordings/ -o ~/transcripts/ \
  --engine openai --lang eng

To use a local Whisper model:

batchalign transcribe ~/recordings/ -o ~/transcripts/ \
  --engine whisper --lang eng

Align transcripts against audio

batchalign align ~/corpus/ -o ~/aligned/

Common useful flags:

batchalign align ~/corpus/ -o ~/aligned/ --engine whisper
batchalign align ~/corpus/ -o ~/aligned/ --utr-engine whisper

Add morphosyntactic analysis

batchalign morphotag ~/corpus/ -o ~/tagged/

Useful variants:

batchalign morphotag ~/corpus/ -o ~/tagged/ --retokenize
batchalign morphotag ~/corpus/ -o ~/tagged/ --keep-existing

Verbosity

batchalign align ~/corpus/ -o ~/aligned/
batchalign -v align ~/corpus/ -o ~/aligned/
batchalign -vv align ~/corpus/ -o ~/aligned/
batchalign -vvv align ~/corpus/ -o ~/aligned/

Next steps

• Batchalign Desktop (Experimental) — in-repo GUI shell status and scope
• CLI Reference
• Performance
• Rev.AI Integration
• Python API

Chatter: tutorials

Learn by completing a guided task.

• Quick Start

Quick Start

Status: Current Last updated: 2026-05-11 17:25 EDT

This page gets you from zero to productive with chatter in five minutes. Install chatter first if you haven’t already.

Validate a CHAT file

Check a single transcript for errors:

chatter validate transcript.cha

If the file is valid:

✓ transcript.cha is valid

If there are problems, you’ll see rich diagnostics with the exact location and a stable error code. For example, an utterance missing its terminator:

  × error[E304]: Expected terminator not found (line 6, column 1)

 6 │ *CHI:	hello world

  help: Add a terminator at the end: Standard (. ? !), Interruption (+... +/.)

Every error code (E304, E705, etc.) links to documentation with fix guidance.

Validate an entire corpus

Point chatter at a directory — it walks recursively, validates in parallel, and caches results:

chatter validate corpus/

The interactive TUI shows progress and lets you browse errors per file. Use --format json for machine-readable output, or --quiet for CI (exit code 1 on errors).

Run an analysis

chatter reimplements the CLAN command surface across validation, analysis, transform, and format-converter categories. A handful of legacy NLP commands (the trie-and-rule MOR/POST/MEGRASP family) are deliberately not implemented — they emit a clear error pointing at Batchalign for the neural replacement. See the CLAN command status matrix for the authoritative per-command counts and per-command status.

Try frequency analysis:

chatter clan freq transcript.cha

Output shows word frequencies by speaker. Add filters:

chatter clan freq transcript.cha --speaker CHI    # one speaker
chatter clan mlu transcript.cha --speaker CHI     # mean length of utterance
chatter clan combo transcript.cha --include-word "want"  # boolean keyword search

All CLAN commands support --format json and --format csv for downstream processing.

Convert to JSON

Get a structured representation of any CHAT file:

chatter to-json transcript.cha

The output conforms to the TalkBank CHAT JSON Schema. Convert back with chatter from-json.

Watch for changes

Edit a file and get live validation feedback:

chatter watch transcript.cha

Every time you save, chatter re-validates and shows updated diagnostics.

What next?

• CLI Reference — all commands, flags, and output formats
• Validation Errors — every error code, with examples and fix guidance
• VS Code Extension — live diagnostics, CLAN commands, media playback
• Migrating from CLAN — flag mapping for CLAN veterans
• Batch Workflows — corpus-scale validation and analysis

VS Code: tutorials

Learn by completing a guided task.

• Your First CHAT File
• Tutorial: Reviewing Aligned Files

Your First CHAT File

Status: Current Last updated: 2026-03-30 13:40 EDT

This chapter walks you through opening a CHAT transcript and exploring the extension’s core features one at a time. Each section introduces a single capability so you can see what the extension does before diving into the detailed reference chapters.

If you have not yet installed the extension, start with Installation.

How the Extension Works

Before we begin, here is a brief look at the architecture. The VS Code extension is a thin TypeScript layer that delegates all CHAT intelligence to a Rust language server. Every feature – validation, alignment, analysis, formatting – runs inside this server.

flowchart LR
    Editor["VS Code Editor\n(TypeScript)"] -->|"stdio LSP"| Server["chatter-lsp\n(Rust binary)"]
    Server --> Parser["Tree-Sitter\nParser"]
    Server --> Model["CHAT Data\nModel"]
    Server --> Validator["Validation\nPipeline"]
    Server --> Analysis["CLAN Analysis\nCommands"]

The editor sends your keystrokes and commands to the server over standard I/O. The server parses the file with a tree-sitter grammar, builds a typed data model, runs validation, and sends diagnostics and responses back. You do not need to know any of this to use the extension – it all happens behind the scenes.

Open a CHAT File

Open any .cha file in VS Code. If you have the talkbank-tools repository cloned, good files to start with are in the resources/corpus/reference/ directory. These files cover a range of CHAT features across 20 languages.

As soon as the file opens, you should see:

• Colored text – headers (@UTF8, @Begin, @Participants) are bold, speaker codes (*CHI:, *MOT:) have their own color, dependent tiers (%mor:, %gra:) have another, and timing bullets are dimmed
• Utterance counts above the @Participants header showing how many utterances each speaker contributes (e.g., “CHI: 42 utterances”)
• A structured outline in the breadcrumb bar and Outline view (Cmd+Shift+O) showing every utterance by speaker

(SCREENSHOT: CHAT file with syntax highlighting) Capture this: a resources/corpus/reference/ file open in the editor. The screenshot should show colored headers, speaker codes, dependent tiers, dimmed timing bullets, and the code lens utterance counts above @Participants.

See Validation Errors

Open the Problems panel with Cmd+Shift+M. If the file has any issues – missing headers, undeclared speakers, alignment mismatches, malformed timing bullets – they appear here as errors and warnings.

Errors also show as red squiggly underlines in the editor text. Warnings show as yellow underlines. Hover over an underlined region to see the diagnostic message and error code.

Validation runs continuously as you type. Make an intentional edit – for example, delete the terminator (. or ?) at the end of an utterance line – and watch a new diagnostic appear within a fraction of a second.

(SCREENSHOT: Problems panel with diagnostics) Capture this: the Problems panel showing 2-3 diagnostics (mix of errors and warnings), with one error also visible as a red underline in the editor.

Play Audio from a Bullet

If the file has an @Media: header pointing to an audio or video file, you can play segments directly from the editor.

Click on a timing bullet – the bullet_number_number markers that look like 15230_18450 (these are millisecond timestamps). Or place your cursor anywhere on a main tier line and press Cmd+Shift+Enter (Play Media at Cursor).

The extension opens a media panel and plays the audio segment corresponding to that utterance.

To play continuously from the cursor to the end of the file, press Cmd+Shift+/ (Play Media Continuously). The editor cursor follows along, highlighting each utterance as it plays.

(SCREENSHOT: Media playback panel) Capture this: the editor with a CHAT file on the left and the media panel open on the right, showing the audio player with playback controls and a speed slider. The current utterance should be highlighted in the editor.

Hover for Cross-Tier Alignment

Place your mouse over any word on a *CHI: or *MOT: main tier line. A hover popup appears showing how that word aligns across all annotation tiers:

• %mor – morphological breakdown (part of speech, lemma, affixes)
• %gra – grammatical relation (SUBJ, OBJ, DET, etc.)
• %pho – phonetic transcription
• %sin – sign/gesture annotation

This alignment works bidirectionally: hover a token on %mor to see the aligned main tier word, or hover a %gra relation to see which word it annotates.

Click any word on any tier and all its aligned counterparts across every other tier highlight simultaneously.

(SCREENSHOT: Cross-tier hover popup) Capture this: the cursor hovering over a word on a *CHI: line, with the hover popup showing aligned %mor and %gra information. Optionally also show the document highlight (colored background) on the corresponding %mor token.

Show the Dependency Graph

Place your cursor on an utterance that has a %gra: tier, then press Cmd+Shift+G (Show Dependency Graph).

A panel opens beside the editor showing the grammatical dependency structure of that utterance as a color-coded graph. Arcs are labeled with grammatical relations (SUBJ, OBJ, DET, MOD, ROOT, etc.) and colored by type.

The graph renders locally using bundled Graphviz WASM – no internet connection required. You can export the graph as SVG or PNG using the toolbar buttons.

(SCREENSHOT: Dependency graph panel) Capture this: the editor with a CHAT file on the left and the dependency graph panel on the right, showing a tree with labeled arcs for a simple utterance like “the dog ate the cookie”.

Run a CLAN Analysis Command

Right-click anywhere in the editor to open the context menu. Under TalkBank: Analysis, select Run CLAN Analysis…

A QuickPick menu appears listing all 33 CLAN analysis commands with one-line descriptions. Select freq (word frequency counts).

The results appear in a styled panel with stat cards, tables, and bar charts. You can export the results to CSV using the “Export CSV” button for use in spreadsheets or statistical software.

Other commonly used commands:

Commands that need extra input (keywords, file paths) prompt you before running. You can also run analysis on an entire directory by right-clicking a folder in the Explorer sidebar and selecting Run CLAN Analysis on Directory…

(SCREENSHOT: CLAN analysis results panel) Capture this: the analysis panel showing freq results – a stat card with total word count, a frequency table, and a bar chart of the top 10 words.

Apply a Quick Fix

If the Problems panel shows an error with a lightbulb icon, the extension can fix it automatically.

Place your cursor on the underlined error and press Cmd+. (Quick Fix). A menu appears with available fixes. For example:

• xx on a main tier line? The fix replaces it with xxx
• Missing terminator? The fix appends the appropriate punctuation
• Undeclared speaker? The fix adds them to @Participants

The extension offers automatic fixes for 21 error codes covering the most common CHAT formatting issues.

(SCREENSHOT: Quick fix menu) Capture this: the cursor on a diagnostic underline, with the quick fix lightbulb menu showing one or two available corrections.

Open the Waveform View

Press Cmd+Shift+W (Show Waveform View) to see a visual representation of the linked audio file.

The waveform panel shows the audio amplitude with colored overlays marking each timed utterance. Click anywhere on the waveform to seek both the audio and the editor cursor to that point. Zoom in and out with the toolbar buttons or mouse wheel.

During playback, the waveform auto-scrolls to keep the current segment visible.

(SCREENSHOT: Waveform panel) Capture this: the waveform panel showing an audio waveform with colored utterance overlays. At least two or three utterance segments should be visible with different colors.

What to Explore Next

You have now seen the core features. Here are some paths to explore further:

• Quick Reference – complete tables of all keyboard shortcuts, commands, and settings
• Media Playback – continuous play, rewind, looping, and playback speed control
• Walker Mode – step through utterances one at a time with Alt+Down / Alt+Up
• Transcription Mode – transcribe from audio with F4 to stamp timing bullets
• Coder Mode – load a codes file and step through uncoded utterances
• Running CLAN Commands – the full set of 33 analysis commands
• Editing Features – validation, quick fixes, completion, snippets, and formatting

Tutorial: Reviewing Aligned Files

Status: Current Last updated: 2026-03-30 13:40 EDT

This tutorial walks through reviewing a CHAT file that was aligned by batchalign align --bullet-repair. It takes about 5 minutes to review a typical file.

Prerequisites

• VS Code Insiders with the TalkBank CHAT extension installed
• A CHAT file aligned with --bullet-repair --review-level=low-confidence
• The corresponding audio file accessible to the extension

Step 1: Open the aligned file

Open the .cha file in VS Code. You should see:

• Yellow left borders on lines with %xrev: [?] — these need review
• The %xalign tiers above them explaining what happened

If you don’t see any yellow lines, the file had no low-confidence decisions. Nothing to review!

Step 2: Start Review Mode

Open the Command Palette (Cmd+Shift+P on macOS, Ctrl+Shift+P elsewhere) and type “Start Review Mode”.

The extension will:

1. Parse all %xrev tiers in the file
2. Show a status bar item: “Review: 0/7” (0 of 7 rated)
3. Jump to the first unreviewed utterance
4. If the waveform panel is open, highlight the bullet segment

Step 3: Listen and rate

For each flagged utterance:

1. Read the %xalign tier — it tells you what happened. For example, boundary_averaged overlap=155ms means two adjacent utterances overlapped and align split the difference.

2. Play the audio — click the bullet (•12095_12335•) or use Cmd+Shift+Enter to play from the current line.

3. Listen — does the audio match the text? Does speech start at the bullet’s timestamp?

4. Rate with a single key:

   • 1 = Good (timing is correct)
   • 2 = Early (audio starts before the speech)
   • 3 = Late (audio starts after the speech)
   • 4 = Wrong (completely wrong location)
   • 5 = Skip (can’t tell)

5. The extension auto-advances to the next unreviewed utterance.

Step 4: Add notes (optional)

If you want to explain your rating, edit the %xrev line directly. Add text after the rating marker:

%xrev:	[early] starts about 200ms before the actual speech

Or:

%xrev:	[wrong] this is a different speaker's audio entirely

Notes are especially valuable when they describe patterns rather than individual utterances:

%xrev:	[early] all backchannels from INV seem to be early by ~200ms

Step 5: Save and done

When the status bar shows all items rated (e.g., “Review: 7/7”), save the file with Cmd+S. The ratings are stored in the %xrev tiers — they’re part of the CHAT file now.

If you use tb-deploy to push changes, the ratings travel with the file.

Step 6: Stop Review Mode

Open the Command Palette and type “Stop Review Mode”, or just close the file. The status bar indicator disappears.

Keyboard shortcut reference

These shortcuts are only active during Review Mode:

The number keys only work when the editor is not focused for text input (i.e., when you’re in the command area or the cursor is on a read-only line). If you need to type in the editor, the number keys behave normally.

Tips

• You don’t need to review every file. Even 5-10 files gives us useful data for tuning the algorithm.

• Focus on the [?] markers. Utterances without %xrev were high-confidence — the algorithm is pretty sure they’re right.

• Pattern notes are gold. “All short backchannels are early” is more useful than rating 50 individual backchannels.

• If unsure, skip. Press 5 to move on. A skipped utterance is better than a wrong rating.

• Review Mode + Walker Mode work well together. Start Review Mode, then use Alt+Down/Alt+Up (Walker) to step through utterances with automatic playback. When you land on a flagged utterance, rate it with 1-5.

What happens with your reviews

The experiment maintainer collects the ratings using a harvesting script and uses them to:

• Evaluate --bullet-repair — is it better than CLAN FIXBULLETS?
• Tune thresholds — how much overlap should be averaged? When should timing be stripped?
• Identify systematic problems — if all backchannels are wrong, the algorithm needs a backchannel-specific fix
• Build ground truth — corrected bullets become training data for improving the alignment algorithm

Your free-form notes are especially valuable — they capture observations that ratings alone can’t express.

CLAN: tutorials

Learn by completing a guided task.

• Quick Start

Quick Start

Status: Current All CLAN commands follow the pattern:

chatter clan <command> [options] <file.cha>

Your first analysis

Given a CHAT transcript sample.cha:

# Word frequency count
chatter clan freq sample.cha

# Mean length of utterance for the child speaker
chatter clan mlu --speaker CHI sample.cha

# Mean length of turn
chatter clan mlt sample.cha

# Vocabulary diversity (D statistic)
chatter clan vocd sample.cha

Working with directories

Process all .cha files in a directory:

chatter clan freq corpus/

Output formats

Every analysis command supports text (default), JSON, and CSV output:

chatter clan freq --format json sample.cha
chatter clan mlu --format csv sample.cha

Filtering

Restrict analysis to specific speakers, tiers, words, or utterance ranges:

# Only the CHI speaker
chatter clan freq --speaker CHI sample.cha

# Exclude the investigator
chatter clan freq --exclude-speaker INV sample.cha

# Only utterances 10-50
chatter clan freq --range 10-50 sample.cha

# Only within gem boundaries
chatter clan freq --gem "story" sample.cha

See the Filtering chapter for the full guide.

Transforms

Transform commands modify CHAT files and write the result:

# Add fluent output tier
chatter clan flo sample.cha

# String replacement from a rules file
chatter clan chstring rules.cex sample.cha

# Reorder dependent tiers
chatter clan tierorder sample.cha

Format conversion

Convert between CHAT and other formats:

chatter clan elan2chat recording.eaf
chatter clan praat2chat recording.TextGrid
chatter clan chat2srt sample.cha

Batchalign: how-to guides

Choose the task you want to complete.

• Install or update Batchalign
• Transcribe a recording
• Align a transcript
• Add morphology and grammar
• Segment utterances
• Add translation tiers
• Compare with a reference
• Configure Rev.AI
• Process Cantonese
• Prepare model downloads
• Manage the result cache
• Tune file concurrency
• Review generated output
• Troubleshoot a run
• Run the desktop from source
• Run the HTTP service
• Migrate a workflow
• Build from source
• Test a change
• Add a command
• Add a backend
• Add language support
• Handle a model-library defect
• Diagnose a pipeline failure
• Release the Python package

Install or update Batchalign

These instructions install the command-line application and its backend extras. The installer also installs uv if needed and selects Python 3.11 for Batchalign.

macOS or Linux

Open Terminal, paste this command, and press Enter:

curl -LsSf https://raw.githubusercontent.com/TalkBank/batchalign/main/bootstrap/bootstrap.sh | sh

Windows

Open PowerShell, paste this command, and press Enter:

powershell -NoProfile -ExecutionPolicy Bypass -Command "Invoke-Expression ((Invoke-WebRequest -UseBasicParsing 'https://raw.githubusercontent.com/TalkBank/batchalign/main/bootstrap/bootstrap.ps1').Content)"

Check the installation

Close and reopen your command window, then run:

batchalign --help
batchalign version

You should see a list of commands and version information. Continue to the Quick Start for your first processing task. If the command cannot be found, see Troubleshooting.

Update an existing installation

Run the same installer again. It upgrades the batchalign[all] tool installation with prereleases allowed, matching the repository’s bootstrap scripts.

For an existing uv installation, the equivalent command is:

uv tool install --upgrade --python=3.11 --prerelease=allow 'batchalign[all]>=0.10'

The package on PyPI is named batchalign. It installs the batchalign command. Model files may download separately when you first use a backend; see Model downloads.

Transcribe a recording

Use transcribe to produce a draft CHAT transcript from a recording. You will need to listen to the recording and correct the draft afterward.

1. Place the recording in a folder you can find, such as recordings.

2. Run the following for English speech:

   batchalign transcribe recordings/ -o transcripts/ --lang eng
   

3. The default engine is Rev.AI. If you have not configured it, follow the credential prompt. Rev.AI processes uploaded audio through your account. See Configure Rev.AI.

4. Wait for the final summary. Open the .cha file in transcripts and review the words, speaker labels, and utterance boundaries against the recording. Generated transcripts include an Unchecked output of ASR model comment.

Use the appropriate three-letter language code, such as spa for Spanish or cmn for Mandarin, instead of eng. --lang is required. Without -o, the transcript is written beside the source recording.

Use a local speech model

To use the Hugging Face Whisper backend:

batchalign transcribe recordings/ -o transcripts/ --lang eng --engine whisper

The first run downloads the selected model. --engine openai selects the local openai-whisper package; it does not mean the hosted OpenAI transcription API. See CLI reference for the engine choices.

Request separate speaker diarization

batchalign transcribe recordings/ -o transcripts/ --lang eng --diarize --num-speakers 2

This adds a separate speaker-assignment stage using the selected diarization backend. Provider speaker labels may already be present without this option.

Add word timings or refine alignment

Use --wor to retain word-level timing output from transcription. For forced alignment of a corrected transcript, run align afterward. Transcription does not automatically run forced alignment or morphosyntactic analysis.

Align a transcript to its recording

Use align after checking the transcript’s words. It adds word-level timing information, including a %wor tier.

1. Put the CHAT transcript and its recording together. Check that @Media: names the recording and @Languages: identifies the transcript language.

2. Run:

   batchalign align corpus/ -o aligned/
   

3. Wait for the final summary. Open the output transcript and check playback against the recording, especially long turns and overlapping speech.

The default forced-alignment engine is wav2vec. When utterance timings are missing, the default timing-recovery engine is Rev.AI; configure its credentials if prompted. Already-timed utterances can avoid that recovery step.

Use local timing recovery

batchalign align corpus/ -o aligned/ --utr-engine whisper

To use Whisper for forced alignment as well:

batchalign align corpus/ -o aligned/ --engine whisper --utr-engine whisper

Use --utr-engine none only when the transcript already has the timings required for forced alignment. Keep different transcript languages in separate alignment invocations: the CLI chooses a backend language from the selected CHAT inputs.

Without -o, source transcripts are updated in place. For model and device options, see CLI reference.

Add morphology and grammar to a transcript

Use morphotag on checked CHAT transcripts to add %mor morphology and %gra dependency tiers. Audio is not required.

1. Put the CHAT files you want to process in a folder. Check that each file’s @Languages: header identifies its language.

2. Choose a separate destination folder, then run:

   batchalign morphotag "corpus" -o "tagged"
   

3. Wait for the final summary. The first use of a language can download Stanza models before tagging begins.

4. Open a transcript in tagged and inspect its %mor and %gra tiers under the speaker lines. Review the analysis before using it in your workflow.

Without -o, this command updates source files in place. Folder input is recursive. You can also supply individual files or an input list.

Retag existing analysis

By default, the CLI removes existing %mor and %gra tiers from staged inputs before processing. Matching backend results may still come from the result cache. To preserve existing analysis and let the engine skip already-tagged utterances:

batchalign morphotag corpus/ -o tagged/ --keep-existing

Allow retokenization

If you need the backend to retokenize text, request it explicitly:

batchalign morphotag corpus/ -o tagged/ --retokenize

Check the resulting main tiers as well as the analysis tiers.

For options, see CLI reference. For reuse and speed, see Caching and Performance.

Segment a transcript into utterances

Use utseg when a CHAT transcript needs utterance boundaries revised.

batchalign utseg corpus/ -o segmented/

Open the output and check the new boundaries against the conversation. Without -o, the command updates source files in place.

The CLI uses the CHAT utterance-segmentation backend. The --language option is passed to that backend and defaults to en. To enable its Stanza fallback:

batchalign utseg corpus/ -o segmented/ --stanza-fallback

See CLI reference for device options. Backend results use the result cache.

Add translation tiers

Use translate to add translation text to CHAT transcripts:

batchalign translate corpus/ -o translated/ --target eng

The default engine is Google. Open the output and review the %eng tiers. The current output tier name remains %eng even when another target language is selected. Without -o, the command updates source files in place.

To select the local NLLB backend:

batchalign translate corpus/ -o translated/ --target eng --engine nllb

The other CLI engine choices are tencent and aliyun. Provider setup and language coverage depend on the selected backend. See CLI reference for options.

Compare a transcript with a reference

Use compare when you have a transcript and a checked reference transcript.

1. Place session.cha and session.gold.cha in the same folder. Alternatively, use template.gold.cha as the reference for files in that folder; a per-file .gold.cha takes precedence.

2. Run:

   batchalign compare corpus/ -o comparisons/
   

3. Inspect the generated comparison metrics and the command’s summary. Gold files are references, not separate inputs to score. Files without a matching reference are skipped by input pairing.

See CLI reference for input selection.

Configure Rev.AI transcription

Rev.AI is the default speech-recognition backend for transcribe and the default utterance-timing recovery backend for align.

Set up your key

Run an interactive transcription command:

batchalign transcribe recordings/ -o transcripts/ --lang eng --engine rev

When prompted, enter your Rev.AI API key. Interactive configuration stores the key in ~/.batchalign.ini, under [asr] as engine.rev.key.

For automation, supply BATCHALIGN_REV_KEY through your environment or secret manager. Quiet mode suppresses interactive credential prompts. The current CLI has no setup subcommand.

Use the configured account

batchalign transcribe recordings/ -o transcripts/ --lang eng --engine rev
batchalign align corpus/ -o aligned/ --utr-engine rev

These operations can upload audio to Rev.AI. Use an account and recordings appropriate for your organization’s workflow. To select a local transcription backend, see Transcribe a recording.

The integration is implemented in the Python backends and configuration module, not a separate Rust server path.

Process Cantonese recordings and transcripts

For a local Cantonese ASR backend, select funaudio and the required language code yue:

batchalign transcribe recordings/ -o transcripts/ --lang yue --engine funaudio

Cloud choices include tencent and aliyun; their credentials and model options are owned by the corresponding Python backend. Use --engine, not the retired JSON engine-override flag. Review the draft words before subsequent analysis.

For morphology, check that the CHAT header contains @Languages: yue, then run:

batchalign morphotag transcripts/ -o tagged/ --retokenize

Retokenization allows word boundaries to change, so inspect the main tiers and analysis tiers in the output. For alignment, follow Align a transcript; model coverage must match the recording and transcript language.

Prepare model downloads

Use this guide before processing on a new computer or before taking a machine offline.

1. Install the backend dependencies using the standard installer.
2. While connected to the internet, run a small representative input with the same command, language, engine, and model you intend to use later.
3. Let the command complete. A successful installation alone does not prove that all required models have downloaded.
4. Repeat for other languages or models you plan to use. A Stanza model for one language does not cover every language.
5. If you need offline operation, test the exact command and input type after disconnecting. Cloud engines still require their external services.

Downloads and disk locations are managed by each backend’s model libraries. Download sizes vary; there is no fixed total size for a Batchalign installation. The current CLI has no model-download or cache-prewarm subcommand.

The Stanza morphosyntax backend refreshes an existing resource catalog once per process, retaining the cached catalog if refresh fails. It loads language pipelines lazily unless a language is pinned at construction. Missing model files can still require downloads. See Performance for the distinction between model files on disk, loaded models in memory, and cached task results.

Manage the result cache

Use these commands when you need to inspect stored results or force a fresh run. They operate on the default result cache, not downloaded models.

Inspect the cache

batchalign cache path
batchalign cache stats

path prints the directory. stats prints its apparent file size and latest modification time, or reports that the cache is absent. It does not count entries.

Clear saved results

Finish or stop processing jobs that use the cache, then run:

batchalign cache clear

Confirm the displayed path when prompted. For an unattended script:

batchalign cache clear --yes

The next processing run recomputes the required backend results and can save them again. Your transcripts, recordings, and downloaded models remain in their own locations.

The current CLI has no per-run cache bypass flag. Python callers can use CacheSpec.bypass() or CacheSpec.refresh().

Tune file concurrency

Use --parallel when you want to change how many files a processing command can work on at once. Start with the default, 8. If memory use is too high, reduce it:

batchalign --parallel 2 morphotag corpus/ -o tagged/

The option goes before the command name. Its minimum is 1. Scripts using the old global --workers option should use --parallel instead.

Compare settings

1. Choose a representative set of files, including the longer transcripts or recordings you normally process.
2. Run with --parallel 1 and a separate output folder. Record elapsed time and peak resident memory with your operating system’s monitoring tools.
3. Repeat with --parallel 4, then 8 if memory permits. Keep the inputs and backend options unchanged.
4. Compare runs with the same cache state. A cached second run is not evidence that a higher concurrency setting improved inference. To compare uncached CLI runs, clear the result cache between trials after each job finishes.
5. Keep the smallest setting that gives useful throughput on your workload.

For morphotag, active files share one Stanza batcher. Increasing --parallel does not create additional Stanza replicas. The default utterance dispatch window and backend batch maximum are both 128; there is no CLI batch-size flag. Python callers can set StanzaBackend(batch_size=..., batch_window_ms=...), but that does not change the Rust per-file dispatch window.

The Python Pipeline and recipe APIs still name file concurrency workers. For the scheduling and memory model, see Performance.

Review generated transcripts and timings

1. Keep the original transcript or recording and write processing output to a separate folder with -o.
2. Read the command’s final summary and inspect failed or skipped files before relying on the output set.
3. For transcription, listen to the recording and correct words, speaker labels, and utterance boundaries. ASR output is marked as unchecked.
4. For alignment, check playback at utterance and word boundaries, especially where speakers overlap or the recording is noisy.
5. For morphology, inspect %mor and %gra against the intended words and language. If retokenization was enabled, review main-tier changes too.

Older files may contain %xalign decision notes or %xrev review markers from other processing versions. Their presence does not prove that the current run created them, and their absence does not prove an output is correct. The current CLI does not expose the old --review-level controls.

Use provenance comments to identify a producing build and task. A file modification timestamp alone cannot establish which utterances were edited or whether a previous review remains valid.

Troubleshoot a processing run

The command cannot be found

Close and reopen Terminal or PowerShell after installation, then try batchalign --help. If it is still unavailable, repeat the installation steps and check that the installer finished.

An option or command is unknown

Run batchalign --help and batchalign COMMAND --help using your installed version. The current global concurrency flag is --parallel; engine selection uses command-specific --engine options. The CLI does not expose the older serve, setup, logs, doctor, or bench interfaces.

A processing command appears to be waiting

The first use of a backend can download models. An interactive cloud backend may also be waiting for credentials. Keep the command window open and read its latest message. Use batchalign -v COMMAND ... for more detail or --plain before the command to avoid live terminal rendering. See Model downloads.

The transcript cannot be parsed

Read the error’s file location and CHAT diagnostic. Fix the indicated transcript and rerun a single file before processing the folder again. For morphology, check @Languages: and the CHAT header structure. Use the CHAT format reference to interpret markup.

Alignment cannot find the recording

Check the transcript’s @Media: header, keep the referenced recording beside the CHAT file, and confirm that the recording is readable. Use a small single-file run to check the pairing. The current local CLI does not use the old server’s remote media mappings.

The computer runs out of memory

Lower file concurrency, for example:

batchalign --parallel 1 morphotag corpus/ -o tagged/

For commands that expose it, --force-cpu selects CPU inference; it is not a morphotag option. Long transcripts and model batch sizes still affect memory at --parallel 1. See Performance.

A repeat run gives the same unexpected result

Check and correct the input first. If you need to rule out saved backend results, finish active processing and clear the result cache, then rerun. Removing model files is a separate operation and is not required to clear saved results.

Report a reproducible problem

Include the output of batchalign version, your operating system, the exact command with credentials removed, and the final error text. Attach a small input that reproduces the problem if you can share it. Say whether it also happens with a fresh result cache. Report issues in the Batchalign repository.

Run the desktop application from source

The Batchalign desktop application lives in apps/batchalign/batchalign-gui/. It uses React and Tauri, with a bundled Python HTTP-service sidecar.

From a development checkout with the repository build tools installed:

just batchalign gui dev

The Bazel dependency graph builds and stages the sidecar. To create a desktop bundle, use just batchalign gui build.

In the application, drop an input folder onto the window or choose open folder, add a command, review its language and engine settings, choose the output location, and start the batch. Folders are searched recursively. The app displays one folder at a time. Use choose… to replace it or close beside the folder chooser to return to the drop-folder screen. The speaker command and Speaker recognition option identify speaker turns. The elapsed timer continues updating while a model is busy.

For transcription, enter a valid three-letter ISO 639-3 language code before choosing an engine. Autocomplete searches codes and language names; Chinese uses zho. No ASR engine is selected automatically. Recommendations appear in a gray box, with the other engines listed below: CHAT-Whisper for English; Aliyun for adult Cantonese and FunASR/SenseVoice for child Cantonese; Paraformer-zh for Mandarin. Google GenAI, OpenAI Whisper, Qwen3, and Rev.AI are the general choices. Rev.AI is always offered after a valid language is entered; other engines follow service availability. Recommendations include cloud and local choices, with cloud engines first. Rev.AI supplies the cloud recommendation when there is no language-specific one; OpenAI Whisper or Qwen3 supplies a local fallback when available. The Recognition engine defaults to pyannoteAI (cloud). The GUI omits Malayalam Wav2Vec, Qwen, Tencent Cloud ASR, and Hugging Face Whisper; their CLI/API availability is unchanged.

For ordinary command-line use, start with the Quick Start. The CLI does not require the desktop service to be running.

Run the HTTP service

Use this guide for API development or a GUI integration. You do not need an HTTP service to process files with the CLI.

1. Install the api extra in your Batchalign environment. The standard bootstrap installation includes the all extra, which includes API dependencies.

2. Start the service in your command window:

   batchalign daemon --host 127.0.0.1 --port 8765
   

3. Open http://127.0.0.1:8765/docs for the API schema, or request http://127.0.0.1:8765/capabilities to discover recipes and backends.

4. Stop the foreground service with Ctrl-C when finished.

Keep the default one Uvicorn worker: the job registry is process-local. The daemon --workers option concerns HTTP processes; it is unrelated to the CLI’s global --parallel option. For supervised operation, run the foreground command under your service manager.

For local path-based API inputs, the service requires BATCHALIGN_API_ALLOW_PATHS=1. Enable it only for a trusted local integration: those paths are interpreted on the machine running the service. The desktop sidecar sets up its own local connection.

The service has no serve start/serve stop command group or server.yaml configuration. See CLI reference for supported options.

Update commands and integrations

1. Update the batchalign package using the installer. Check batchalign version before comparing behavior.
2. Replace the former global --workers N flag with --parallel N. It controls active files, not model-process replicas.
3. Use command-specific --engine options for ASR and FA. Check the actual choices with batchalign COMMAND --help; old engine names are not all interchangeable.
4. Use -o/--out for a destination. Processing commands accept multiple paths and -i/--input-list/--file-list.
5. Remove old automatic-server, fleet, benchmark, and cache-override flags. Ordinary CLI processing now runs in process. The separate daemon is an HTTP service for GUI/API integrations.
6. Update cache assumptions: text and audio backend results use LMDB. See Caching before comparing repeat-run performance.
7. For Python integrations, use the current Pipeline, recipe, input, and backend interfaces. The old BA2 document/pipeline classes are not drop-in equivalents.

Run a small copied input through the revised workflow before processing a corpus. The CLI reference lists the supported surface.

Build Batchalign from source

Use the repository’s just recipes from the checkout root. They invoke Bazel and materialize the Rust extension, Python dependencies, and generated protocol types required by each target.

just batchalign build debug
just batchalign test debug
just batchalign cli --help

just batchalign cli is the authoritative development CLI entrypoint. It restores the invocation directory before running the Bazel Python binary, so relative input paths behave as they do in an installed CLI.

Build release artifacts

just batchalign wheel
just batchalign sidecar
just batchalign gui build

wheel produces the host-platform Python wheel in python/target/wheels/. sidecar builds the PyApp-bundled HTTP service. The GUI bundle embeds that sidecar. Cross-platform wheel builds run in the GitHub Actions matrix.

Run checks

just batchalign test
just batchalign pytest
just batchalign lint
just batchalign gui test
just docs build

See Testing for scope. just batchalign versions shows the version sources; just --list batchalign lists recipes. Use the repository wrappers instead of a separate editable-install procedure for ordinary development.

Test a Batchalign change

Run commands from the repository root using just.

For a focused Python case:

just batchalign pytest -k cli_input_list

For an integration check, use the real development entrypoint:

just batchalign cli --parallel 2 morphotag /path/to/fixtures -o /tmp/tagged-check

Choose inputs that exercise the behavior being changed. For inference changes, compare semantic outputs as well as elapsed time and RSS; record cache state and model settings. Tests with fake backends check orchestration, not model accuracy.

The product recipe explicitly includes engine unit, cache-smoke, and pipeline-smoke targets that a broad wildcard can omit because of manual tags. The wheel CI matrix also checks native platform packaging and Python 3.10 imports.

GUI checks need a browser installed for the pinned Playwright version. On a cold machine, the bundled runtime may need first-launch setup before it can serve requests. Distinguish setup failures from frontend assertions when diagnosing results.

Add a processing command

1. Identify the input/output types and task sequence. Reuse a recipe in python/batchalign/recipes.py or add one that constructs Pipeline.
2. Implement a command module under python/batchalign/cli/. Use resolve_inputs for multiple paths and file lists, and cli_options(ctx) for global settings.
3. Pass opts.parallel as the recipe’s workers value. Use Interface for progress and the shared output helpers for outcomes where their contract fits.
4. Register the module in _LAZY_SUBCOMMANDS in cli/__init__.py. A file under cli/hidden/ is not automatically a public command.
5. Test help, argument validation, output destinations, and failures. Exercise a real invocation through just batchalign cli.
6. Add a task-focused how-to page and update the CLI reference.

morphotag.py demonstrates staged CHAT preprocessing; convert.py demonstrates output collision checks. Use the example that matches the new command’s behavior. See Pipeline architecture for the runtime.

Add an inference backend

1. Choose the task interface in python/batchalign/backends/base.py. Implement the corresponding backend marker, a stable name, its BatchPolicy, and call behavior. Start from an existing backend for the same task.
2. Return the typed output expected by the core proto/ definition. Preserve input/output correspondence and return one output per input in a batch.
3. Include output-affecting model/configuration changes in backend identity when they are not already in the typed input. The engine uses that name in cache keys.
4. Export the backend through the package’s lazy import surface where appropriate, add its dependencies to the relevant python/pyproject.toml extra, and supply it to a recipe. For a CLI engine choice, update that command’s enum and builder.
5. Test protocol shape and failure behavior with deterministic fixtures, then perform a small real-backend smoke run through just batchalign cli.
6. Update the command reference and any backend-specific instructions.

Batching and result caching are engine responsibilities. Do not add a worker process or a second cache solely to integrate an ordinary Python backend. A genuinely new task also needs core input/output types, a runner, and routing registration; inspect the existing task family before extending those contracts.

See Source map, Python API, and Testing.

Add or change language support

1. Identify the command and backend you are extending. Language-code recognition, model availability, and useful analysis quality are separate checks.
2. Check python/batchalign/lang.py and the backend’s vendor-code conversion. For Stanza, inspect backends/morphosyntax/ud/lang.py and stanza.py.
3. Test representative text or audio with the selected model. Include the language’s contractions, script, punctuation, and conversational forms.
4. For ASR, check number expansion and cleanup in the Rust core asr/ modules. For morphology, check token reconciliation and typed tier alignment as well as the model’s raw output.
5. Add regression fixtures at the responsible layer and run the appropriate tests, followed by a small CLI integration run.
6. Update language selection and the language-specific reference. Record measured quality with model versions and corpus scope instead of inferring support from a language list.

Handle a model-library defect

1. Reduce the failure to a small reproducible input and record library/model versions and configuration.
2. Determine whether the error originates in the model, a vendor adapter, token reconciliation, or CHAT result application.
3. Add a regression at that boundary. Test a nearby valid case so a workaround does not merely suppress a class of outputs.
4. Implement a narrowly scoped correction where the required information is available. Stanza token processing/rendering lives in the Python backend; typed CHAT injection lives in the Rust morphology runner.
5. Re-run the affected command on representative inputs and compare semantic output. Record known limits rather than claiming a universal linguistic fix.
6. If behavior affects cached outputs, update backend identity as appropriate; compiled build identity also participates in result-cache keys.

Historical model probes are useful evidence, but rerun a relevant case against the current pinned environment before treating an old mitigation table as active.

Diagnose a pipeline failure

1. Reproduce with one small input through just batchalign cli, using a separate output directory. Preserve the original failing input.
2. Add global -v or -vv and capture the command’s error text. Use --plain when saving terminal output to a file.
3. Determine whether failure occurs during input parsing, backend construction, backend inference, result application, or output writing. Use the source map to find the owning layer.
4. Compare cached and fresh behavior when relevant using the Python cache policy or the CLI cache-management command after active processing stops.
5. Add a focused regression case at the layer that owns the defect. A fake backend can isolate scheduler and result-injection behavior from model behavior.

The current CLI does not expose the retired --debug-dir, server job logs, or worker-protocol trace commands. Inspect Outcome.error, progress callbacks, and the Python/Rust logging surfaces used by the current pipeline. Include version, command, input shape, and cache state in a reproducible report.

Release the Python package

1. Set the Python package version in python/pyproject.toml. The repository wrapper refreshes python/uv.lock; check the resulting diff. Rust workspace and desktop versions have separate owners.
2. Run the relevant checks, inspect their results, and record any known baseline failures separately from regressions.
3. Commit the intended release state and push it to the branch CI will build.
4. Dispatch .github/workflows/publish-pypi.yml with tag exactly matching the Python version. The workflow builds native macOS ARM/Intel, Linux ARM/x86-64, and Windows x86-64 wheels.
5. Verify that every artifact came from the intended commit, has the expected version/platform tag, and passes bazel/python/verify_wheel.py and twine check.
6. Publish through the configured channel. The workflow’s publish=true route requires PyPI trusted publishing. If that is not configured, build with publish=false, download all five wheel artifacts, and upload them using authenticated Twine.
7. Verify PyPI’s file list and hashes against the artifacts you uploaded.

Do not publish local development wheels in place of the matrix artifacts. The exact workflow and package metadata are the release contract; historical batchalign3 distribution and Rust-server release instructions do not apply.

Chatter: how-to guides

Choose the task you want to complete.

• Installation
• Migrating from CLAN
• VS Code Extension
• Chatter Desktop (Experimental)
• Batch Workflows
• CI Integration
• CHAT Processing Playbook (Editors & Analysts)
• Sanitize (Protected Corpora)

Installation

Status: Current Last updated: 2026-06-01 00:50 PDT

chatter runs on Windows, macOS, and Linux. Pre-built binaries are available from the GitHub Releases page; choose the plain vX.Y.Z TalkBank core release entry and its chatter-v... assets (VS Code preview releases use vscode-vX.Y.Z tags in the same repository). To build from source, follow the instructions below.

Prerequisites

• Rust (stable) — install via rustup (supports Windows, macOS, Linux)

If you are only building the chatter CLI from source, Rust is sufficient. The extras below are only needed when you work on the grammar or generated artifacts:

• Node.js — required for tree-sitter grammar generation
• tree-sitter CLI — cargo install tree-sitter-cli

Installing chatter (CLI)

The chatter CLI is the primary tool for working with CHAT files. It provides validation, format conversion, and batch processing.

From Source

Clone the repository:

mkdir -p ~/talkbank && cd ~/talkbank
git clone https://github.com/TalkBank/batchalign.git

Build and install:

cd ~/talkbank/batchalign
cargo install --path crates/chatter/chatter-cli

This installs the chatter binary to ~/.cargo/bin/ (macOS/Linux) or %USERPROFILE%\.cargo\bin\ (Windows).

Verify Installation

chatter --version
chatter --help

Building the Libraries

If you’re developing with the Rust crates directly:

cd ~/talkbank/batchalign
cargo build           # Build all crates
cargo test            # Run all tests
cargo clippy          # Lint check

See the Makefile targets for additional commands.

Directory Layout

Everything lives in a single repository:

~/talkbank/
└── batchalign/             # This repository
    ├── grammar/            # Tree-sitter grammar
    ├── crates/             # All Rust crates (talkbank-* and batchalign-*)
    ├── resources/spec/     # CHAT specification (source of truth)
    ├── resources/corpus/   # Reference corpus
    ├── apps/vscode-extension/   # VS Code extension
    ├── apps/chatter/       # Chatter Desktop (Tauri, experimental)
    ├── apps/batchalign/    # Batchalign Desktop + dashboard (Tauri, experimental)
    ├── python/             # batchalign Python package
    └── book/               # TalkBank Toolchain mdBook

The CLI, grammar, crates, Batchalign pipeline, and all related tools are in this single repository.

Migrating from CLAN to talkbank-tools

Status: Current Last updated: 2026-05-11 16:38 EDT

This page is for two audiences:

• users moving day-to-day workflows from legacy CLAN binaries to chatter
• developers moving implementation work from older CLAN- or Java/Python-based tooling to the Rust rewrite

It is not a promise of character-for-character equivalence with every historical CLAN binary. The goal of the rewrite is a stable, documented Rust toolchain for CHAT parsing, validation, normalization, JSON interchange, and CLAN-style analyses.

User Workflow Crosswalk

Legacy Flag Translation

chatter accepts both legacy CLAN syntax (+t*CHI, -t*MOT, +s"word", +z25-125) and the modern explicit syntax (--speaker CHI, --exclude-speaker MOT, etc.). Both work identically — you do not have to retrain muscle memory. Under the hood, a pre-clap pass in clan_core::clan_args::rewrite_clan_args rewrites any +flag/-flag tokens that follow the clan subcommand into their --flag equivalents before argument parsing proper begins. Anything the rewriter does not recognise passes through unchanged, so mistyped legacy flags surface as ordinary clap errors rather than silent misinterpretation.

The table below is the complete list of legacy flags the rewriter currently handles. If a flag does not appear here, it is not translated; use the modern --flag spelling.

CHECK (chatter clan check …) has its own context-sensitive overloads. Inside the check subcommand the rewriter recognises these additional forms:

+g1–+g5 are only rewritten as CHECK generics when the check subcommand is present in the argument list; in every other command, +g… remains gem filtering.

Common CLAN Workflows in chatter

Side-by-side examples for the most frequent CLAN tasks. Either the legacy or the modern form works — pick whichever is easier to read.

Frequency analysis for one speaker:

# Legacy CLAN
freq +t*CHI file.cha

# chatter, modern syntax
chatter clan freq file.cha --speaker CHI

# chatter, legacy syntax (rewritten internally)
chatter clan freq file.cha +t*CHI

MLU for a speaker over a utterance range:

# Legacy
mlu +t*CHI +z25-125 file.cha

# chatter
chatter clan mlu file.cha --speaker CHI --range 25-125

KWAL search with context window:

# Legacy
kwal +t*CHI +s"cookie" -w2 +w3 file.cha

# chatter
chatter clan kwal file.cha \
    --speaker CHI --include-word cookie \
    --context-before 2 --context-after 3

FREQ with multiple filters:

# Legacy
freq +t*CHI +t%mor -s"the" file.cha

# chatter
chatter clan freq file.cha --speaker CHI --tier mor --exclude-word the

CHECK restricted to a single error code:

# Legacy
check +e6 file.cha

# chatter
chatter clan check file.cha --error 6

Gem-scoped analysis:

# Legacy
freq +gstory +t*CHI file.cha

# chatter
chatter clan freq file.cha --gem story --speaker CHI

What’s Different

Practical changes to expect when you swap a legacy CLAN tool for chatter:

• Output formats. chatter commands accept --format text|json|csv|clan where supported; legacy CLAN produced only its own text layout. Build pipelines on --format json rather than parsing text.
• Error codes. Diagnostics use the stable E### / W### system documented in book/src/operations/errors/, not CLAN’s varied error numbering.
• Cache. chatter validate memoises clean results in the OS cache directory and reuses them on subsequent runs. Pass --force to bypass the cache.
• Unicode. Full UTF-8 throughout the pipeline; CLAN had several encoding quirks that chatter does not reproduce.
• Determinism. The same input always produces the same output. A handful of legacy CLAN commands had timing- or filesystem-order-dependent behaviour that chatter intentionally does not preserve.
• JSON interchange. chatter to-json / chatter from-json roundtrip through a published JSON schema. CLAN had no stable interchange format.
• Platform. One binary builds and runs on Windows, macOS, and Linux. No more platform-specific CLAN binaries or encoding quirks.
• Unknown legacy flags fail loudly. If the rewriter does not recognise a +x or -y token, it is passed through to clap unchanged, which then reports a normal argument error. There is no silent acceptance of unsupported flags.

Important Behavioral Differences

• chatter clan is one command family, not a directory of separate CLAN executables.
• JSON conversion is first-class and schema-backed.
• Validation diagnostics use Rust error codes like E301 or W602, not CLAN’s historical numbering.
• Cache behavior is explicit. validate reuses cached clean results unless --force is passed.
• Normalization does not overwrite files unless you give an output path yourself.

Recommended User Migration Steps

1. Start by replacing corpus-wide CHECK runs with chatter validate.
2. Switch any JSON/export glue to chatter to-json and chatter from-json.
3. Migrate CLAN analysis invocations to chatter clan ... and verify any output-format assumptions.
4. Rebaseline automation on Rust error codes and structured JSON output rather than parsing legacy text output.

Developer Crosswalk

What to Change in Developer Workflows

• Build on the Rust crates instead of editing CLAN-era text manipulation logic.
• Treat the AST, serializer, and validator as the source of truth for current behavior.
• Add or update integration tests when changing CLI-visible behavior or corpus-wide semantics.
• Document behavior changes in this book when they affect users or downstream integrations.

Scope of Compatibility

talkbank-tools aims for practical migration, not blind historical reproduction:

• legacy data should continue to parse where feasible
• CLAN-style analyses should document any deliberate divergence
• new documented public contracts are the Rust CLI, preview source-first Rust crates, JSON Schema, and documented diagnostics

VS Code Extension

Status: Current Last updated: 2026-05-11 18:58 EDT

Support tier: Public preview. VSIX bundles are published on GitHub Releases, not yet available from the VS Code Marketplace.

A full-featured CHAT editor for VS Code built on the Rust-powered chatter-lsp language server. Edit, validate, analyze, transcribe, and play back audio — all inside your editor. Leverages the same CHAT engine as the chatter CLI with live real-time feedback.

Platform support: Windows, macOS, and Linux

Installation

The extension is located in apps/vscode-extension/. Two install paths, depending on what you want to do.

End users — install a released VSIX. Pre-built, platform-specific VSIX bundles (each ships with the matching chatter-lsp binary) are published on GitHub Releases. Download the .vsix for your platform, then in VS Code: Extensions sidebar → “…” menu → Install from VSIX… and pick the downloaded file.

Developers — run from source. Open the apps/vscode-extension/ folder in VS Code, install dependencies, compile, and launch the Extension Development Host:

cd vscode
npm install
npm run compile
# Then press F5 in VS Code to launch the dev host

The dev host runs the locally-compiled extension and talks to a chatter-lsp binary discovered via the resolver described in the Configuration and Architecture sections below.

Validation & Diagnostics

Live Validation

Errors and warnings appear inline as you type — the same error codes as chatter validate (currently 191; verify with rg -c '#\[code\("[EW]' crates/core/talkbank-model/src/errors/codes/error_code.rs), with source-level precision:

• Squiggly underlines on the exact error location
• Problems panel integration with error codes and descriptions
• Validates on every keystroke (250ms debounce), plus immediately on open and save
• Filter by severity: all (default), errors only, or errors and warnings

Quick Fixes

Press Cmd+. (Mac) / Ctrl+. (Windows/Linux) to auto-fix common issues. Supports 21 error codes including:

• Add missing terminators, fix xx → xxx, insert undeclared speakers
• Swap reversed timestamps, close unclosed brackets, delete empty utterances
• Fix consecutive stress markers, consecutive commas, trailing-off markers

Inlay Hints

Inline annotations show alignment mismatches (e.g., [alignment: 3 main ↔ 2 mor]) and timing durations directly in the editor. Toggle with talkbank.inlayHints.enabled.

Validation Explorer

A tree view in the Explorer sidebar for bulk validation of entire directories. Runs the chatter validate CLI under the hood, caches results, and displays errors as navigable tree nodes. Right-click any folder to validate.

Cross-Tier Alignment Visualization

Hover

Hover over any word on the main tier to see its aligned morphological breakdown (%mor), grammatical relations (%gra), phonological form (%pho), and aligned items from %sin — all in a rich Markdown popup.

Hover works bidirectionally: hover a %mor item to see the aligned main-tier word.

Highlight Linking

Click any word on the main tier and all aligned items on dependent tiers light up. Click a %mor or %gra item and the corresponding main-tier word highlights. Bidirectional across %mor, %gra, %pho, %sin, %mod.

Dependency Graph

Cmd+Shift+G (Mac) / Ctrl+Shift+G (Windows/Linux) renders the %gra dependency tree for the utterance at your cursor. Words appear as labeled nodes with colored edges by relation type (subject, object, root, etc.). Rendered via Graphviz WASM — no internet required.

Toolbar: zoom, fit to window, export as SVG or PNG.

Media Playback

Single Segment

Cmd+Shift+Enter (Mac) / Ctrl+Shift+Enter (Windows/Linux) plays the timing bullet (•beg_end•) nearest the cursor. Audio/video resolves from the @Media: header.

Continuous Playback

Cmd+Shift+/ plays all segments from cursor to end of file. The editor cursor tracks the currently-playing utterance, auto-advancing through the transcript.

Playback Controls

Waveform View

Cmd+Shift+W opens an interactive waveform panel. Each timing bullet appears as a colored rectangle over the audio waveform. Click anywhere on the waveform to seek. Zoom in/out with buttons, slider, or mouse wheel. Auto-scrolls during playback.

Transcription Mode

A dedicated workflow for transcribing audio into CHAT format.

1. Start: talkbank.startTranscription — opens the media panel with the full audio file
2. Play: Audio plays from your cursor position
3. Stamp: Press F4 to insert a timing bullet at the current playback position and open a new *SPEAKER: line
4. Rewind: Press F8 to jump back and re-listen
5. Stop: talkbank.stopTranscription — exits transcription mode

Configure the default speaker code (talkbank.transcription.defaultSpeaker, default: “CHI”) and rewind duration.

Walker Mode

Step through utterances one at a time with synchronized audio playback.

Each step moves the editor cursor and optionally plays the corresponding audio segment. Configure auto-play, loop count (0 = infinite), pause between segments, and walk length.

CLAN Analysis Commands

Run any of the 34 CLAN analysis commands directly from VS Code — no terminal needed (see status-matrix.md for the authoritative list).

Right-click a .cha file or folder → Run Analysis → pick a command from the list:

• Frequency/Count: freq, mlu, mlt, wdlen, wdsize, maxwd, freqpos, timedur, vocd, …
• Syntax: complexity, dss, ipsyn, sugar, …
• Search: kwal, combo, …
• Assessment: kideval, eval, eval-d

Results appear in a dedicated panel with styled tables, stat cards, bar charts, and a CSV export button.

KidEval / Eval / Eval-D

Dedicated assessment panels for child language evaluation, general evaluation, and dementia assessment. Select language and normative database, filter by age and gender, and compare z-scores against population means.

Navigation

Editing

Code Completion

• Speaker codes (from @Participants)
• Dependent tier prefixes (%mor, %gra, %pho, etc.)
• Header names (triggers on @)
• Bracket annotations (triggers on [)
• Postcode punctuation

Snippets

8 built-in CHAT snippets: file header, @Participants, @ID, main tier, %mor, %gra, @Comment, Gem (@Bg/@Eg).

Participant Editor

talkbank.editParticipants opens a table editor for all @ID headers. Edit the 10 pipe-delimited fields (language, corpus, speaker, age, sex, group, SES, role, education, custom) in a form view. Changes are written back as canonical @ID lines.

Linked Editing

Edit a speaker code and all matching codes in the file update simultaneously.

Auto-Formatting

Format document rewrites to canonical CHAT format. On-type formatting auto-indents after tier prefixes.

Special Characters

Compose Mode

Type CA (Conversation Analysis) and CHAT special symbols without memorizing Unicode:

Press a key to insert the symbol. Press Escape to cancel. Status bar shows the active mode.

Coder Mode

Annotate transcripts with coding schemes from .cut files.

1. Start coder mode (context menu → “Start Coder Mode”)
2. Load a .cut codes file with hierarchical codes
3. Step through uncoded utterances (Cmd+Enter / Ctrl+Enter)
4. Insert codes from a hierarchical picker (Cmd+Shift+C / Ctrl+Shift+C)
5. Codes appear as %cod: tiers

Progress tracking shows how many utterances remain uncoded.

Speaker Filtering

Context menu → “Filter by Speaker…” opens a multi-select picker. Choose one or more speakers to create a filtered read-only view showing only those speakers’ utterance blocks, with all file headers preserved.

Code Lens

Utterance counts per speaker appear above the @Participants header (e.g., CHI: 42 utterances).

Additional Features

Configuration

Architecture

The extension launches the dedicated chatter-lsp binary over stdio using the Language Server Protocol. (chatter-lsp is a separate Rust crate at crates/chatter/chatter-lsp/; the VS Code extension’s executable resolver at apps/vscode-extension/src/executableService.ts looks for it in the bundled extension binary, then on PATH, then under the extension’s target directory.) The LSP server provides:

• Incremental parsing via tree-sitter
• Full validation (currently 191 error codes; see live count above)
• Cross-tier alignment computation
• CLAN analysis command execution
• Document formatting and symbol extraction

Interactive features (media playback, waveform, graphs, analysis panels, assessment tools) run as VS Code Webview panels with HTML/CSS/JS. The Graphviz renderer uses WASM — no external dependencies.

Development

cd vscode
npm install          # Install dependencies
npm run compile      # Build the extension
npm run watch        # Watch mode for development
npm test             # Run tests
npm run lint         # ESLint check

To debug: open the apps/vscode-extension/ folder in VS Code and press F5 to launch the Extension Development Host.

Chatter Desktop (Experimental)

Status: Current Last updated: 2026-06-01 00:50 PDT

⚠️ IMPORTANT: Chatter Desktop is an experimental in-repo evaluation app, NOT a supported end-user surface. Use the chatter CLI for production CHAT validation, or use the VS Code extension for interactive editing with live validation.

When to use Chatter Desktop

Chatter Desktop (apps/chatter/chatter-gui/) is intended only for:

• Contributors and maintainers evaluating the native GUI implementation
• Local development and testing of the desktop surface
• Experimental internal workflows

For supported end-user workflows:

• Validate CHAT from the command line: use chatter validate
• Edit CHAT with live diagnostics: use the VS Code extension

This is also not the Batchalign desktop app. The two experimental desktop surfaces in this repository are:

• Chatter Desktop (apps/chatter/chatter-gui/) — CHAT validation GUI only (experimental)
• Batchalign Desktop (apps/batchalign/batchalign-gui/) — Batchalign processing UI (experimental)

Current status

• Release contract: Experimental (see repo-root book/src/operations/release-contract.md)
• Distribution: No supported DMG/MSI/AppImage release channel
• CI: Not included in main PR CI gates
• Recommendation: Contributors and maintainers only; do not use for production work

Getting Started (Development Only)

From source

cd apps/chatter/chatter-gui
npm install
cargo tauri dev       # launches the app with hot reload
cargo tauri build     # produces a distributable app bundle

Requires: Rust (stable, edition 2024), Node.js, and npm.

Using the App

Opening files

Chatter validates one target at a time: a single .cha file or one folder.

Three ways to start validating:

1. Choose File — opens a file picker filtered to .cha files
2. Choose Folder — opens a folder picker; validates all .cha files recursively
3. Drag and drop — drag one .cha file or one folder onto the app window

When idle, if you’ve previously validated a target, the drop zone shows “Last: resources/corpus/reference/ — Re-validate?” as a clickable shortcut.

Reading results

The main window has three areas:

┌──────────────────────────────────────────────────────────────┐
│  [Choose File] [Choose Folder] or drag here  [System|Light|Dark] │
├──────────────────┬───────────────────────────────────────────┤
│ 3 FILES WITH     │  Filter by code… [All|Errors|Warnings]    │
│ ERRORS / 120     │                                           │
│                  │  ▾ [E302] Missing @End header              │
│  📁 corpus/      │  ┌───────────────────────┐                │
│    ✗ file1 (3)   │  │ 41 │ *CHI: hello .    │                │
│    ✗ file3 (1)   │  │ 42 │                   │                │
│                  │  │    │ ^                 │                │
│                  │  └───────────────────────┘                │
│                  │  💡 Add @End on the last line             │
│                  │  [Copy] [Open in CLAN]                    │
├──────────────────┴───────────────────────────────────────────┤
│  Progress: 45/120 │ 4 errors │ ~2m 30s remaining │ [Cancel]  │
└──────────────────────────────────────────────────────────────┘

• File tree (left) — collapsible directory tree showing only files with errors (valid files are hidden to reduce clutter). A header shows “N files with errors / M total”. Files are sorted alphabetically.

• Error panel (right) — for the selected file, shows each error with its code in [E001] format, severity color, message, source snippet with caret underlines, and multi-span labels for complex errors (e.g., alignment mismatches across tiers). CHAT-specific formatting is handled: tabs expanded to 8-column boundaries, \x15 bullets rendered as •, underline markers shown as styled underlined text. Suggestions prefixed with 💡.

• Status bar (bottom) — streaming progress during validation, ETA after 5+ files, total error count, and action buttons.

Filtering errors

A compact filter bar appears above the error cards when a file has diagnostics:

• Code filter — type “E7” to show only alignment errors, “W” for warnings, etc.
• Severity toggle — switch between All / Errors / Warnings

The file header updates to show filtered vs. total count (e.g., “3 errors (7 total)”).

Collapsible error cards

Each error card has a clickable header that toggles between expanded and collapsed view. Collapsed cards show only the error code and first line of the message. When a file has 5 or more errors, an Expand All / Collapse All button appears.

Dark mode

Chatter follows your system appearance by default. A System / Light / Dark toggle in the drop zone area lets you override. Your preference is remembered across sessions.

The dark palette uses muted Apple-style colors — readable miette error highlighting on dark backgrounds.

Clickable file paths

Click the file name in the error panel heading to reveal the file in Finder (macOS), Explorer (Windows), or the default file manager (Linux).

Copy errors

Each error card has a Copy button that copies the full miette-rendered error text (plain text, not HTML) to your clipboard for pasting into issue reports or messages.

Actions

“Open in CLAN” only appears when the CLAN application is detected on your system (macOS and Windows only). It adjusts line numbers to account for headers that CLAN hides (@UTF8, @PID, @Font, @ColorWords, @Window).

Keyboard shortcuts

All other navigation is mouse-driven (click files, scroll errors).

Window title

The window title updates to reflect the current state:

• Idle: “Chatter”
• Discovering: “Chatter — Discovering files…”
• Running: “Chatter — Validating (45/120)”
• Finished: “Chatter — 14 errors in 3 files” or “Chatter — All 74 files valid”

ETA

After 5 or more files have been processed, the status bar shows an estimated time remaining (e.g., “~2m 30s remaining”). The estimate updates every second.

Notifications

When validation finishes while the app is not focused, a system notification shows the summary (“Validation complete — 14 errors in 3 files”).

First launch

On first launch, an onboarding overlay explains the four main interactions: drag files, error panel, keyboard shortcuts, and export. Dismiss with “Got it” — it won’t appear again.

CLI Bundling

The desktop app can bundle the chatter CLI binary so power users who download the GUI can also run the CLI from their terminal (like VS Code ships the code command).

An Install CLI Command menu item (when available) symlinks the bundled binary to /usr/local/bin/chatter (macOS/Linux) or copies it to a PATH directory (Windows).

To build with the bundled CLI:

cargo build --release -p chatter-cli
mkdir -p apps/chatter/chatter-gui/src-tauri/resources
cp target/release/chatter apps/chatter/chatter-gui/src-tauri/resources/
cargo tauri build

Architecture

The desktop app lives in apps/chatter/chatter-gui/ as a sibling to apps/vscode-extension/:

apps/chatter/chatter-gui/
  src-tauri/          Rust backend (Tauri v2)
    src/
      main.rs         Bin entry — calls chatter_gui_lib::run()
      lib.rs          Tauri app setup (Builder + module wiring)
      protocol.rs     Shared command/event names + request types
      commands.rs     validate, cancel, open_in_clan, export, reveal, install_cli
      events.rs       ValidationEvent → frontend event bridge
      validation.rs   Desktop validation orchestration for one target
  src/                React + TypeScript frontend
    components/       DropZone, FileTree, ErrorPanel, ProgressBar, OnboardingOverlay
    hooks/            useValidation, validationState, useTheme
    protocol/         Command/event names + TypeScript transport mirrors
    runtime/          Tauri transport + capability-focused runtime seam

The Rust backend calls validate_directory_streaming() from talkbank-transform directly — the same streaming validation pipeline used by the TUI. Events flow over crossbeam channels to the Rust side, then are serialized to JSON and emitted to the frontend via Tauri’s event bridge.

Cancellation uses ArcSwapOption for lock-free atomic swap of the cancel sender — no mutex.

The frontend keeps Tauri-specific code confined to src/runtime/tauriTransport.ts. React components and hooks consume narrower capabilities (validationRunner, validationTarget, clan, exports) instead of reaching for one broad desktop service object.

Comparison with TUI

Both use the identical validation engine and produce the same error codes.

When to Use Which Tool

The TalkBank toolchain offers validation through four interfaces. Each serves a different workflow:

Chatter Desktop focuses on validation only — it does not run CLAN analysis commands or play media. If you need those features, use the VS Code extension.

Batch Workflows

Status: Current Last updated: 2026-05-02 02:30 EDT

The chatter CLI is designed for processing large CHAT corpora efficiently. This page covers common batch workflows.

Validating a Corpus

Validate all .cha files in a directory tree:

chatter validate /path/to/corpus/

The validator recursively discovers .cha files and processes them in parallel. Results are cached — subsequent runs skip unchanged files.

Forcing Revalidation

To bypass the cache and revalidate everything:

chatter validate /path/to/corpus/ --force

Filtering Output

Show only errors (hide warnings):

chatter validate /path/to/corpus/ --quiet

Stop after the first reported error:

chatter validate /path/to/corpus/ --max-errors 1

Write a JSONL audit file while validating:

chatter validate /path/to/corpus/ --audit validation.jsonl

CHAT-JSON Roundtrip

Convert an entire corpus to JSON and back:

# CHAT → JSON
for f in corpus/**/*.cha; do
  chatter to-json "$f" > "${f%.cha}.json"
done

# JSON → CHAT
for f in corpus/**/*.json; do
  chatter from-json "$f" > "${f%.json}.roundtrip.cha"
done

The roundtrip is designed to preserve the ChatFile model. In regression tests, compare normalized output rather than assuming byte-for-byte identity after parser or serializer changes.

Cache Management

The validation cache stores results for previously validated files (keyed by content hash). The cache database file is named talkbank-cache.db and lives in the OS cache directory:

• macOS: ~/Library/Caches/talkbank-chat/talkbank-cache.db
• Linux: ~/.cache/talkbank-chat/talkbank-cache.db
• Windows: %LocalAppData%\talkbank-chat\talkbank-cache.db

It can hold results for large file collections.

chatter cache stats    # Show hit rates and entry count
chatter cache clear --all

Do not delete the cache file manually while chatter is running.

Reference Corpus Validation

The talkbank-tools repository includes a reference corpus at resources/corpus/reference/ (currently ~100 .cha files; verify by find resources/corpus/reference -name '*.cha' | wc -l). The parser must handle every file in this corpus at 100%:

cargo nextest run -p talkbank-parser-tests -E 'test(parser_equivalence)'

This runs the parser equivalence test — each .cha file is its own test, so nextest runs them in parallel and reports individual failures.

Integration with batchalign

The Batchalign pipeline uses the same Rust core (via PyO3) for CHAT parsing and serialization. Since the 2026-04-28 monorepo merge, Batchalign source lives inside this repository under crates/batchalign-* (the standalone batchalign3 GitHub repo was archived). Files processed by Batchalign produce valid CHAT that passes chatter validate.

CI Integration

Status: Current Last updated: 2026-04-13 19:23 EDT

How to use chatter in continuous integration pipelines.

Exit Codes

All examples below rely on exit code 1 to signal validation failure.

Basic Usage

chatter validate corpus/ --quiet --tui-mode disable

• --quiet suppresses per-file success output
• --tui-mode disable prevents interactive TUI (required in non-TTY environments)
• Exit code 0 means all files valid; 1 means errors found

GitHub Actions Example

- name: Validate CHAT corpus
  run: |
    chatter validate corpus/ --quiet --tui-mode disable --format json --audit results.jsonl

- name: Upload validation report
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: validation-report
    path: results.jsonl

The --audit results.jsonl flag streams per-error JSON lines to a file, which is useful for archiving or downstream analysis even when the step fails.

JSON Output for Automation

chatter validate corpus/ --format json --tui-mode disable 2>/dev/null

Each file produces a JSON object on stdout with status, error_count, and errors array. The exit code still reflects overall pass/fail.

Pre-commit Hook

#!/bin/sh
# .git/hooks/pre-commit
chatter validate . --quiet --tui-mode disable

This blocks commits that introduce invalid CHAT files. The hook runs quickly on cached files; only modified files are re-validated.

Suppressing Specific Errors

Some corpora have known issues that should not block CI. Use --suppress to ignore specific error codes or named groups:

chatter validate corpus/ --suppress E726,E727,E728 --tui-mode disable

Or use the named group shorthand:

chatter validate corpus/ --suppress xphon --tui-mode disable

Suppressed errors do not appear in output and do not affect the exit code.

Audit Mode for Large Corpora

For bulk corpus validation where you want a full error database without caching overhead:

chatter validate corpus/ --audit errors.jsonl --tui-mode disable

The --audit flag streams one JSON object per error to the specified file. A summary is printed to stderr at the end.

CHAT Processing Playbook for Editors and Analysts

Status: Current Last updated: 2026-03-24 00:01 EDT

Objective

Provide practical guidance for non-compiler users who create, edit, and validate CHAT files, with emphasis on error interpretation and correction workflow.

Who This Is For

• Transcript editors,
• corpus curators,
• QA reviewers,
• linguists using tooling outputs but not parser internals.

Core Editing Workflow

1. Open file in editor with CHAT diagnostics enabled.
2. Run validation (single file first, then batch).
3. Fix highest-severity structural issues first (headers, tier markers, unmatched delimiters).
4. Re-run validation and inspect warnings.
5. Only then address style and normalization suggestions.

Error Triage Heuristic

• Errors at file start: likely header formatting or encoding issues.
• Errors at tier prefix: likely malformed */% tier syntax.
• Errors inside words: likely symbol, marker, or annotation boundary issues.
• Repeated same error class: likely one systemic rule violation pattern.

Fast Interpretation Guide

• Error: parser/validator could not accept structure; must fix.
• Warning: valid but suspicious or non-canonical; review strongly recommended.
• Info: advisory normalization or convention hints.

Common Fix Recipes

• Header spacing problems:
  • Ensure expected separators and avoid accidental tabs/spaces drift.
• Unclear language/form markers:
  • Confirm @s usage and suffix ordering with house style guide.
• Duration/annotation confusion:
  • Verify bracketed annotation form and avoid malformed punctuation.
• Dependent tier attachment issues:
  • Ensure % tiers follow intended main tier and keep indentation consistent.

Batch Validation Workflow

1. Validate a small sample first.
2. Group failures by error code.
3. Fix by pattern, not file-by-file random order.
4. Re-run and confirm error count decreases monotonically.
5. Save run report for audit trail.

Collaboration Workflow with Developers

When reporting parsing issues, include:

• exact file path,
• minimal excerpt around failing span,
• observed diagnostic code/message,
• expected behavior (if known).

This reduces back-and-forth and speeds defect triage.

Quality Checklist Before Publishing Corpus Updates

• No unresolved error-level diagnostics.
• Warning classes reviewed and accepted or fixed.
• Participant headers and IDs internally consistent.
• Roundtrip serialization check passes for representative samples.
• Changelog note recorded for major normalization edits.

Training Recommendations

• Maintain short examples for each common error class.
• Provide editor cheat sheet for tier prefixes and marker syntax.
• Run periodic QA calibration sessions across editors.

Sanitize (chatter debug sanitize)

Status: Current Last updated: 2026-04-28 22:18 EDT

chatter debug sanitize strips contributor lexical content from a CHAT file while preserving structure (timing bullets, %wor per-word offsets, speaker codes, dependent-tier scaffolding, structural counts, POS tags, language markers). Output is structurally identical to the input but contains no participant words, names, or free-text annotations.

The command exists so engineering tooling — including LLM-assisted debugging — can operate on protected-corpus files (aphasia/, dementia/, rhd/, fluency/Password/, clinical-children corpora, etc.) without exposing contributor speech to commercial LLM services.

When to use it

Run chatter debug sanitize on the source file before loading it into any tool (LLM-backed debugger, scratch directory, screen-shareable session) where you don’t want participant content visible.

When you need to ask a contributor for help debugging a specific file, frame the request as “run the sanitizer locally and send me the output” rather than asking for the raw file.

Usage

# Write sanitized output to stdout
chatter debug sanitize input.cha

# Write sanitized output to a file
chatter debug sanitize input.cha --output sanitized.cha

Working location for sanitized files: prefer a stable, non-/tmp scratch directory (e.g. set TB_SCRATCH_DIR to a per-project dir under your workstation’s persistent storage) for any state that should outlive a single command. macOS clears /tmp on reboot.

What is preserved (byte-exact)

• Timing bullets •start_end• on the main tier.
• %wor per-word offsets (word START_END triples).
• Speaker codes (*PAR, *INV, *CHI, …).
• Utterance count, word count per utterance, dependent-tier count.
• Structural markers: compound +, clitic ~, CA elements, overlap points, lengthening, stress markers, syllable pause, underline begin/end, proper-noun @n markers.
• Language markers (@s:LANG), form types (@a, @b), POS tags ($adj, $n).
• Headers: @Languages, @Birth, @Date, @Media, @PID, @L1Of, @Begin/@End/@UTF8.
• %mor POS categories and morphological features (e.g., n|, -Past).
• %gra (numeric grammatical relations) and %tim (timing).
• Untranscribed tokens xxx / yyy / www — preserving them changes semantic meaning, so they pass through unchanged.

What is replaced or redacted

Determinism + Idempotence

Placeholder generation is keyed off (utterance_index, word_index) tree position, not a global counter. Two consequences:

• Deterministic: sanitizing the same input twice produces byte-identical output.
• Idempotent: sanitizing a sanitized file produces the same file again — no double-replacement, no shifting placeholder numbers.

Pipeline

flowchart LR
    Input["Source .cha\n(protected corpus)"] --> Parser["TreeSitterParser\n(talkbank-parser)"]
    Parser --> Model["ChatFile model\n(talkbank-model)"]
    Model --> Sanitize["sanitize()\n(talkbank-transform::redact)"]
    Sanitize --> Walker["walk_words_mut\n+ header walker\n+ dep-tier walker\n+ scoped-annot walker"]
    Walker --> Mutated["Mutated ChatFile\n(placeholders + redactions)"]
    Mutated --> Writer["WriteChat\n(byte-exact bullets)"]
    Writer --> Output["Sanitized .cha\n(scratch path)"]

The walker step replaces WordContent::Text segments inside Word.content, mutates MorWord.lemma fields, redacts free-text header / dep-tier / scoped-annotation strings, and drops phonological tiers. WriteChat then re-serializes — and because it serializes from Word.content (not from Word.raw_text), every CA element, compound marker, clitic boundary, and timing bullet round-trips byte-exact.

Out of v1 scope

Documented for transparency; v2 work:

• Speaker-code anonymization (graph rewrite across @Participants, @ID, *SPK:, @Birth, @L1Of).
• @Birth / @Date fuzzing (exact birth dates can be identifying).
• @Media filename redaction.
• Audio-side sanitization. (Audio bytes are never touched by the sanitizer; the audio stays at its original path.)
• “Unsanitize” or round-trip mapping. Explicitly not built — the sanitizer is one-way, the mapping table that would reverse it is the exact artifact we don’t want to exist.

Implementation

Library module: talkbank_transform::redact. CLI surface: chatter debug sanitize. The strict policy is the only public preset in v1; future variants can grow on SanitizationPolicy.

VS Code: how-to guides

Choose the task you want to complete.

• Installation
• Syntax Highlighting
• Real-Time Validation
• Quick Fixes
• Code Completion & Snippets
• Special Characters
• Participant Editor
• Document Symbols
• Go to Definition
• Cross-Tier Alignment
• Dependency Graphs
• Speaker Filtering
• Scoped Find
• Media Playback
• Waveform Visualization
• Walker Mode
• Transcription Mode
• Media Resolution
• Running CLAN Commands
• Profiling Commands
• Frequency & Distribution
• Assessment Tools
• Command Reference
• Rating Utterances
• Interactive Bullet Correction
• Harvesting Results
• Codes Files (.cut)
• Coding Workflow
• Corpus Validation
• Transcription from Audio
• Post-Alignment Review
• Batch Processing
• Common Issues
• LSP Connection
• Media Not Found
• Architecture
• Adding Features
• Custom Commands
• Testing
• Releasing

Installation

Status: Current Last updated: 2026-06-01 00:52 PDT

This chapter walks you through installing VS Code and the TalkBank CHAT extension. The language server (chatter-lsp) ships inside each platform-specific VSIX, so there is no separate binary to install. By the end, you will have a working environment where opening any .cha file gives you syntax highlighting, real-time validation, and all the features described in this book.

Prerequisites

You need:

1. Visual Studio Code Insiders version 1.110 or later
2. A platform-specific .vsix package — each release publishes five VSIXes (darwin-arm64, darwin-x64, linux-x64, linux-arm64, win32-x64) with a prebuilt chatter-lsp bundled inside

The extension is a .vsix package installed into VS Code. Each platform-specific VSIX bundles the matching chatter-lsp binary at <extension>/server/chatter-lsp[.exe], so end users do not need a Rust toolchain or any PATH setup.

Step 1: Install VS Code Insiders

macOS

Install via Homebrew:

brew install --cask visual-studio-code@insiders

This gives you the code-insiders command in your terminal.

Windows

Download from https://code.visualstudio.com/insiders/. Run the installer and ensure “Add to PATH” is checked so the code-insiders command is available from the terminal.

Linux

Download the .deb or .rpm package from https://code.visualstudio.com/insiders/, or use snap:

sudo snap install code-insiders --classic

Step 2: Install the Extension

Option A: Platform-specific VSIX (recommended)

1. Open https://github.com/TalkBank/batchalign/releases

2. Choose the vscode-vX.Y.Z release entry and download the .vsix that matches your host:

   • talkbank-chat-darwin-arm64.vsix (Apple Silicon Macs)
   • talkbank-chat-darwin-x64.vsix (Intel Macs)
   • talkbank-chat-linux-x64.vsix
   • talkbank-chat-linux-arm64.vsix
   • talkbank-chat-win32-x64.vsix

3. Install it:

   code-insiders --install-extension talkbank-chat-<platform>.vsix
   

   Or from inside VS Code: open the Command Palette (Cmd+Shift+P), type “Extensions: Install from VSIX…”, and select the .vsix file.

Each platform VSIX bundles a matching chatter-lsp binary. No Rust toolchain or PATH setup is required.

Option B: Development mode (building from source)

Contributors running from a cloned workspace can build chatter-lsp and launch the extension directly:

# From the talkbank-tools/ repository root:
cargo build --release -p chatter-lsp

cd apps/vscode-extension
npm install
npm run compile     # or: just vscode build (from repo root)

# Launch VS Code with the extension loaded in development mode:
code-insiders --extensionDevelopmentPath=.

In this mode the extension picks up the dev-tree build at ../target/release/chatter-lsp (or target/debug/ after cargo build -p chatter-lsp). Producing a full set of platform VSIXes is covered in the Releasing chapter.

Step 3: Verify the Installation

1. Open VS Code Insiders
2. Open any .cha file (if you have the repository, try one from resources/corpus/reference/)
3. You should see:
   • Syntax highlighting – headers, speaker codes, dependent tiers, and annotations are colored by category
   • Validation diagnostics – any errors or warnings appear as squiggly underlines and in the Problems panel (Cmd+Shift+M)
   • “CHAT Validation” tree view in the Explorer sidebar

(SCREENSHOT: VS Code Insiders with a CHAT file open) Capture this: a .cha file from resources/corpus/reference/ open in the editor, showing syntax highlighting, the Problems panel with diagnostics, and the CHAT Validation tree view in the Explorer sidebar.

If you see plain, uncolored text, the extension did not activate. Check:

• The file has a .cha extension
• The language mode in the bottom status bar says “CHAT” (not “Plain Text”)
• The Output panel (Cmd+Shift+U) shows a “TalkBank Language Server” channel with server startup messages

How the LSP Binary Is Found

When the extension starts, it searches for the chatter-lsp binary in this order:

1. talkbank.lsp.binaryPath setting – if set and the file exists, that path is used directly (explicit override)
2. Bundled binary – <extension>/server/chatter-lsp[.exe], shipped with each platform-specific VSIX
3. System PATH – which chatter-lsp, for developers who install via cargo install
4. Dev-tree builds – <extension>/../target/debug/chatter-lsp or .../target/release/chatter-lsp, for contributors running from a cloned workspace
5. If none of these find a binary, an error notification appears with a link to the talkbank.lsp.binaryPath setting

The binary is spawned with no arguments. It speaks the Language Server Protocol over stdio as soon as it starts.

See Settings Reference for the talkbank.lsp.binaryPath setting, and LSP Connection for troubleshooting a missing binary.

Next Steps

With the extension installed, open a CHAT file and follow along with Your First CHAT File to explore the core features.

Syntax Highlighting

Status: Current Last updated: 2026-03-30 13:40 EDT

The extension provides two layers of syntax highlighting for CHAT files: a TextMate grammar for immediate fallback coloring, and LSP semantic tokens for context-aware precision. Together they make CHAT transcripts readable at a glance, with distinct colors for headers, speaker codes, annotations, morphology, and other structural elements.

Two Highlighting Layers

TextMate Grammar (Fallback)

The TextMate grammar (syntaxes/chat.tmLanguage.json) provides basic token coloring that activates instantly when you open a .cha file, before the language server has finished starting. It uses pattern-matching rules to identify CHAT constructs by their textual form.

LSP Semantic Tokens (Context-Aware)

Once the language server is running, it provides semantic tokens that override the TextMate grammar with more precise coloring based on the parsed AST. For example, the semantic layer can distinguish specific POS tags in morphology, identify error-coded tokens, and differentiate between header types that look similar as raw text.

The language server advertises 11 semantic token types:

Range-based semantic tokens are supported, computing tokens only for the visible range. This reduces work for large files.

Color Categories

The following table summarizes how CHAT elements are colored in the editor. Exact colors depend on your VS Code color theme, but the semantic categories are consistent:

On-Type Formatting

When you type : on a *SPEAKER: or %tier: line, a tab character is automatically inserted after the colon. This matches the CHAT format convention where a tab always follows the colon on speaker and tier lines. No manual tab insertion is needed.

This behavior is provided by the language server’s textDocument/onTypeFormatting handler and works in real time as you type.

Document Formatting

Format the entire document to canonical CHAT style with Shift+Alt+F (or your configured format shortcut), or via the Command Palette (Format Document).

The formatter normalizes:

• Whitespace and indentation
• Header ordering
• Speaker code formatting
• Tier indentation (tabs after colons)
• Line endings

If the document is already in canonical form, no changes are applied. You can also enable Format on Save via VS Code’s editor.formatOnSave setting to automatically normalize the file every time you save.

The formatting operation is backed by the language server’s textDocument/formatting handler, which serializes the document through the canonical CHAT serializer. This ensures the output matches the authoritative CHAT format exactly.

Related Chapters

• Real-Time Validation – error squiggles work alongside highlighting
• Code Completion & Snippets – completions appear in the context of highlighted elements
• Quick Fixes – fix errors that are highlighted as diagnostics
• Cross-Tier Alignment – clicking highlighted words shows cross-tier connections

Real-Time Validation

Status: Current Last updated: 2026-04-16 22:27 EDT

The extension validates your CHAT file continuously as you type. Errors appear as underlined squiggles with diagnostic messages in the Problems panel — no need to save or run a separate command.

What gets validated

Diagnostic structure

Each diagnostic includes:

• Error code (e.g., E301, E705) — links to the error reference
• Severity — Error (red), Warning (yellow), Information (blue), Hint (gray)
• Related information — points to the conflicting location (e.g., the @Participants header when a speaker is undeclared)

(SCREENSHOT: Editor showing validation errors with squiggles and Problems panel) Capture this: Open a CHAT file with deliberate errors (missing @End, undeclared speaker). Show the red squiggles and the Problems panel at the bottom.

Inlay hints

When the main tier and %mor tier have different word counts, inlay hints appear as muted inline annotations showing the alignment status.

When counts match, a checkmark appears. When they mismatch, the hint shows both counts with a cross mark, making it easy to spot alignment issues without running a separate command.

Toggle with talkbank.inlayHints.enabled setting.

Diagnostic tags

Some diagnostics use VS Code’s diagnostic tag system for visual styling:

• Unnecessary — fade-out (dimmed text) for deprecated constructs
• Deprecated — strikethrough for obsolete syntax

Severity filtering

The talkbank.validation.severity setting controls which diagnostics appear:

Validation pipeline

flowchart TD
    Change["Document Change"] --> Debounce["250ms Debounce"]
    Debounce --> Parse["Parse\n(tree-sitter)"]
    Parse --> Validate["Model Validation\n(~190 error codes)"]
    Validate --> Convert["Convert to\nLSP Diagnostics"]
    Convert --> Publish["Publish to\nProblems Panel"]

Changes are debounced at 250ms to avoid overwhelming the parser during rapid typing. After parsing, the full model validation pipeline runs (the same pipeline as chatter validate), producing diagnostics with error codes, spans, and related information.

See also

• Quick Fixes — automatic fixes for common errors
• Corpus Validation — validating entire directories

Quick Fixes

Status: Current Last updated: 2026-03-30 13:40 EDT

When the language server detects certain errors, it can offer automatic corrections. These appear as a lightbulb icon in the left gutter, or you can invoke them directly with Cmd+. (macOS) / Ctrl+. (Windows/Linux). Quick fixes are the fastest way to resolve common CHAT formatting and structural problems without manual editing.

How Quick Fixes Work

1. The language server continuously validates the document as you type (see Real-Time Validation).
2. For errors that have a well-defined automatic correction, a code action is registered.
3. When your cursor is on a diagnostic that has a quick fix, a lightbulb icon appears in the gutter.
4. Click the lightbulb or press Cmd+. to see the available fixes.
5. Select a fix to apply it instantly.

Some errors offer multiple fix options (for example, E301 offers three different terminators). Others apply a single unambiguous correction.

(SCREENSHOT: Quick fix lightbulb showing options for E308 (undeclared speaker)) Capture this: open a .cha file with a speaker code on a main tier that is not listed in @Participants. Place cursor on the error squiggle, then press Cmd+. to show the lightbulb menu with the “Add ‘INV’ to @Participants” option.

Available Quick Fixes

The extension currently provides automatic fixes for 21 error codes:

Diagnostic Tags

Some quick-fixable errors use VS Code’s fade-out styling to visually indicate that content can be safely removed. Empty utterances (E306) and empty colons (E322) appear with dimmed text in the editor, making them easy to spot as candidates for deletion.

Tips

• Batch fixing: When a file has many instances of the same error (for example, multiple undeclared speakers), you can fix them one at a time with Cmd+. or use the Problems panel (Cmd+Shift+M) to navigate between them.
• Undo: Every quick fix is a normal text edit, so Cmd+Z reverts it if the fix was not what you intended.
• Format after fixing: Some fixes (like inserting a missing @Begin) may leave whitespace that is not perfectly canonical. Use Document Formatting (Shift+Alt+F) to normalize the file after applying fixes.

Related Chapters

• Real-Time Validation – how diagnostics are produced
• Syntax Highlighting – visual indicators for CHAT elements
• Code Completion & Snippets – proactive insertion of correct CHAT structures

Code Completion & Snippets

Status: Current Last updated: 2026-03-30 13:40 EDT

The extension provides context-aware autocompletion and snippet templates to speed up CHAT editing. Completions are triggered by specific characters at the start of a line or within an utterance, and snippets expand common CHAT structures with a single Tab press.

Code Completion

Completion suggestions appear automatically as you type certain trigger characters. You can also invoke the completion menu manually with Ctrl+Space.

Speaker Codes (trigger: *)

When you type * at the start of a line, the extension offers all speakers declared in the file’s @Participants header. Each suggestion includes the speaker’s role and auto-inserts the colon and tab separator:

*|  <-- cursor here

Suggestions:
  CHI   target child     --> inserts "*CHI:\t"
  MOT   mother           --> inserts "*MOT:\t"
  FAT   father           --> inserts "*FAT:\t"

This ensures you never mistype a speaker code or use a code that is not declared in the file.

Tier Types (trigger: %)

When you type % at the start of a line after a main tier, the extension offers all standard dependent tier types:

%|  <-- cursor here

Suggestions:
  mor   Morphological analysis       --> inserts "%mor:\t"
  gra   Grammatical relations        --> inserts "%gra:\t"
  pho   Phonological transcription   --> inserts "%pho:\t"
  mod   Model phonology              --> inserts "%mod:\t"
  sin   Gesture/sign annotations     --> inserts "%sin:\t"
  act   Action coding                --> inserts "%act:\t"
  cod   General coding               --> inserts "%cod:\t"
  com   Comments                     --> inserts "%com:\t"
  ...and more

Headers (trigger: @)

When you type @ at the start of a line, the extension offers all 28 standard CHAT headers with descriptions. Each suggestion auto-inserts the header name, colon, and tab separator:

(SCREENSHOT: Completion menu showing @ header options) Capture this: in a .cha file, place cursor at the beginning of a blank line in the header section and type @. The completion menu should appear showing headers like @Languages, @Participants, @ID, @Media, @Date, @Location, etc.

Standard headers include @Begin, @End, @UTF8, @Languages, @Participants, @ID, @Media, @Date, @Location, @Situation, @Activities, @Comment, @Bg, @Eg, @New Episode, @Birth of, @Birthplace of, @Education of, @Group of, @SES of, @Sex of, @Warning, @Transcriber, @Coder, @Options, @Font, @ColorWords, and @Window.

Brackets (trigger: [)

When you type [ inside an utterance, the extension offers 17 standard bracket annotations with descriptions:

• [//] – retracing with correction
• [/] – retracing without correction
• [//?] – uncertain retracing
• [*] – error marking
• [= text] – explanation
• [: alt] – replacement
• [=! text] – paralinguistic material
• [+ post] – postcode
• [- pre] – precode
• [%act: text] – inline action
• [%com: text] – inline comment
• And others

This eliminates the need to memorize all bracket codes and ensures correct formatting.

Postcodes (trigger: +)

When you type + in an utterance context, the extension offers valid CHAT postcodes:

+|  <-- cursor here

Suggestions:
  +"     Quotation follows
  +,.    Self-completion
  +/.    Interruption
  +/?    Interruption question
  +//.   Self-interruption
  +...   Trailing off

Code Lens

Above the @Participants header, a code lens annotation displays the utterance count for each speaker in the file. For example:

CHI: 42 utterances | MOT: 38 utterances
@Participants:	CHI Target_Child, MOT Mother

This provides an at-a-glance summary of speaker activity without running an analysis command. The counts update automatically as you edit the document.

Code Folding

The extension provides CHAT-aware code folding via the language server:

• Utterance blocks: Each main tier line (*SPEAKER:) together with its dependent tiers (%mor:, %gra:, etc.) folds as a single unit. Collapsing an utterance hides all its annotation tiers while keeping the main tier visible.
• Header block: The header section (@Begin through the first utterance) folds as one unit, letting you hide metadata to focus on the transcript body.

Use the fold/unfold controls in the gutter, or the keyboard shortcuts:

• Fold: Cmd+Shift+[ (macOS) / Ctrl+Shift+[ (Windows/Linux)
• Unfold: Cmd+Shift+] (macOS) / Ctrl+Shift+] (Windows/Linux)
• Fold All: Cmd+K Cmd+0
• Unfold All: Cmd+K Cmd+J

Snippets

The extension includes 8 CHAT snippets for common file structures. Type the prefix and press Tab to expand. Tab stops (shown as $1, $2, etc. in the template) let you fill in values sequentially by pressing Tab.

Example: Using the Header Block Snippet

1. Open a new .cha file
2. Type newfile and press Tab
3. The full file skeleton expands with cursor positioned at the language code
4. Press Tab to move through each placeholder: language, participant names, ID fields, first utterance content

Snippets are defined in snippets/chat.json and are available in any .cha file.

Related Chapters

• Quick Fixes – automatic corrections for errors
• Syntax Highlighting – visual feedback while editing
• Special Characters – inserting CA and CHAT Unicode symbols
• Participant Editor – visual table editor for @ID headers

Special Characters

Status: Current Last updated: 2026-03-30 13:40 EDT

CHAT transcription uses many Unicode symbols that are not available on a standard keyboard – overlap markers, prosody arrows, IPA-adjacent characters, and specialized punctuation. The extension provides a compose-key system for inserting these characters without memorizing Unicode codepoints or relying on system input methods.

How Compose Mode Works

1. Press the compose shortcut to enter a mode:
   • Cmd+Shift+1 (macOS) / Ctrl+Shift+1 (Windows/Linux) for CA mode (Conversation Analysis symbols)
   • Cmd+Shift+2 (macOS) / Ctrl+Shift+2 (Windows/Linux) for CHAT mode (CHAT-specific symbols)
2. A status bar indicator appears showing the active compose mode (e.g., “CA Char…” or “CHAT Char…”).
3. Press the trigger key corresponding to the symbol you want to insert.
4. The Unicode symbol is inserted at the cursor position, and compose mode deactivates.
5. Press Escape at any time to cancel without inserting anything.

(SCREENSHOT: Status bar showing compose mode active) Capture this: press Cmd+Shift+1 in a .cha file. The status bar at the bottom of the VS Code window should show a “CA Char…” indicator. Capture the status bar region.

This compose-key system is ported from the CLAN macOS application’s CharToSpChar() function, so researchers familiar with CLAN’s character input will find the same key mappings.

CA Mode Characters (Cmd+Shift+1)

Conversation Analysis notation uses specialized Unicode symbols for prosody, overlap, voice quality, and other phonetic features. After pressing Cmd+Shift+1, press one of the following keys:

Overlap Markers

The four overlap bracket symbols (⌈ ⌉ ⌊ ⌋) mark the boundaries of overlapping speech between two speakers. In CA transcription:

• Speaker A’s overlap is marked with ceiling brackets: ⌈overlapping words⌉
• Speaker B’s overlap is marked with floor brackets: ⌊overlapping words⌋

Prosody Contours

The five numbered keys (1-5) map to pitch contour arrows arranged from highest to lowest, matching standard CA prosody notation conventions.

CHAT Mode Characters (Cmd+Shift+2)

CHAT format uses additional special characters for phonological notation, grouping, and morphological analysis. After pressing Cmd+Shift+2, press one of the following keys:

Tips

• One character at a time: Compose mode deactivates after each character insertion. To insert multiple special characters in sequence, press the compose shortcut again before each one.
• Mode confusion: If you are unsure which mode you are in, check the status bar indicator. Press Escape to cancel and start over.
• System input methods: On macOS, you can also use the built-in Character Viewer (Ctrl+Cmd+Space) as an alternative for occasional use. The compose-key system is faster for frequent transcription work.

Related Chapters

• Syntax Highlighting – how special characters are colored in the editor
• Code Completion & Snippets – bracket annotations that contain some of these symbols
• Quick Fixes – automatic corrections for common CHAT errors

Participant Editor

Status: Current Last updated: 2026-03-30 13:40 EDT

CHAT files encode participant metadata in @ID headers as pipe-delimited strings with 10 fields. Editing these by hand is tedious and error-prone – it is easy to miscount pipes, leave a field blank, or use an invalid format. The Participant Editor provides a visual table interface for viewing and editing all participant information at once.

Opening the Editor

There are two ways to open the Participant Editor:

• Right-click in a .cha file and select Edit Participants… from the context menu
• Command Palette (Cmd+Shift+P / Ctrl+Shift+P) and search for TalkBank: Edit Participants

A webview panel opens beside the editor showing all @ID lines in a tabular form.

(SCREENSHOT: Participant editor panel with filled fields) Capture this: open a .cha file that has at least two @ID headers (e.g., one for CHI and one for MOT). Open the participant editor via right-click. The panel should show a table with rows for each participant and columns for all 10 fields. Fill in some values (age, sex, role) so the table is not empty.

Fields

The table displays one row per participant with 10 editable columns, corresponding to the 10 pipe-delimited fields of the @ID header:

Editing and Saving

1. Click any cell in the table to edit its value.
2. Modify as many fields as needed across any number of participants.
3. Click Save Changes to write the updated @ID lines back to the document.

When you save, the editor generates canonical @ID header lines and replaces the existing ones in the .cha file. The formatting is handled entirely by the language server – the TypeScript UI layer sends the edited data to the LSP via talkbank/formatIdLine, and the server produces correctly formatted pipe-delimited output.

Architecture

The Participant Editor is a thin UI layer over two LSP commands:

• talkbank/getParticipants – parses all @ID headers in the current document and returns the structured field data.
• talkbank/formatIdLine – takes edited field data and serializes it back to a canonical @ID header string.

No CHAT parsing happens in TypeScript. This ensures that the editor always produces correctly formatted output that matches the language server’s understanding of the document.

Tips

• Adding a new participant: Add the speaker to @Participants first (either manually or via the E308 quick fix), then open the Participant Editor to fill in the @ID fields.
• Speaker code renaming: If you need to change a speaker code, use the Rename feature (F2) instead, which updates the code across @Participants, @ID, and all main tier lines in one operation.
• Age format: CHAT ages follow the format years;months.days – for example, 2;6. means 2 years and 6 months (days omitted). The editor does not enforce this format, so take care to use it correctly.

Related Chapters

• Code Completion & Snippets – the @ID snippet provides a quick template for new ID headers
• Quick Fixes – E308 adds undeclared speakers to @Participants
• Real-Time Validation – validation checks participant consistency

Document Symbols

Status: Current Last updated: 2026-03-30 13:40 EDT

The extension provides standard VS Code symbol navigation features – document outline, workspace search, rename, find references, linked editing, and code lens – all aware of CHAT structure. These make it easy to navigate large transcript files and perform safe, file-wide speaker operations.

Document Symbols (Outline)

Press Cmd+Shift+O (macOS) / Ctrl+Shift+O (Windows/Linux) to open the document symbol picker. The outline shows:

• @ headers – @Begin, @Participants, @ID, @Languages, @Media, and all other headers
• Speaker lines – every *SPK: main tier line, grouped under the speaker code

This is the same data that populates the Outline view in the Explorer sidebar. Use it to jump directly to any header or speaker utterance.

(SCREENSHOT: Document outline showing headers and speaker lines in the Outline sidebar view) Capture this: open a .cha file with multiple speakers and several headers. Show the Outline view in the Explorer sidebar with the hierarchical symbol tree.

Workspace Symbols

Use Cmd+T (macOS) / Ctrl+T (Windows/Linux) to search across all open CHAT files for headers and speaker lines. Type a speaker code like CHI or a header name like @Media to jump to any matching location across the entire workspace, without switching files manually.

This is especially useful when working with a corpus of related transcripts and you need to find a specific speaker or header across multiple files.

Rename Speaker (F2)

Place your cursor on a speaker code – either on a *CHI: main tier line or in the @Participants header – and press F2. Type the new code and all occurrences are updated atomically:

• The speaker entry in @Participants
• The corresponding @ID header line
• All main tier lines using that speaker code (*CHI: becomes *NEW:)

This is a standard LSP rename operation. It also works via right-click Rename Symbol or the Command Palette (TalkBank: Rename Symbol).

(SCREENSHOT: Rename dialog on a speaker code showing the preview of all affected locations) Capture this: place cursor on *CHI and press F2. Type a new code like TAR and show the rename preview with all locations that will change.

Find All References (Shift+F12)

Place your cursor on a speaker code and press Shift+F12 (or right-click Find All References). The References panel shows every location where that speaker appears:

• @Participants declaration
• @ID header line
• All main tier lines for that speaker

This helps you quickly see the full extent of a speaker’s contributions and verify that all references are consistent.

Linked Editing

When you edit a speaker code, all other occurrences of that speaker in the document are highlighted for simultaneous editing. As you type, every matching instance updates in real time.

This is different from F2 Rename: linked editing is live, in-place, character-by-character editing of all matching speaker codes. F2 Rename is a one-shot atomic operation with a preview dialog.

Enable linked editing via VS Code’s editor.linkedEditing setting:

{
  "editor.linkedEditing": true
}

Code Lens

Above the @Participants header, a code lens annotation shows the utterance count for each speaker (e.g., CHI: 42 utterances | MOT: 38 utterances). This provides an at-a-glance summary of speaker activity without running an analysis command.

The counts update automatically as you edit the file.

Related Chapters

• Go to Definition – jump from a speaker reference to its @Participants declaration
• Speaker Filtering – view only selected speakers’ utterances
• Cross-Tier Alignment – hover and highlighting for tier-level navigation
• Keyboard Shortcuts – customize symbol navigation bindings

Go to Definition

Status: Current Last updated: 2026-03-30 13:40 EDT

Press F12 or Cmd+Click (macOS) / Ctrl+Click (Windows/Linux) on a speaker code or dependent tier item to jump to its definition or aligned source. The extension knows the alignment between main tier words, %mor items, and %gra relations, so it can navigate across tiers in a single keystroke.

Speaker Definition

Clicking on a speaker code (e.g., *CHI on a main tier line, or CHI inside a %gra or %mor tier) jumps to its declaration in the @Participants header. This is the canonical location where the speaker’s full name and role are defined.

%mor to Main Tier

Clicking on a %mor item jumps to the aligned word on the main tier. For example, clicking n|cookie on the %mor tier jumps the cursor to the word cookie on the *CHI: line above.

The alignment is computed from the parsed model’s positional correspondence between main tier words and morphological items, not from string position guessing.

(SCREENSHOT: Go-to-definition from %mor tier to main tier) Capture this: Cmd+Click on a %mor item like n|cookie. The cursor should jump to the corresponding word cookie on the main tier line above.

%gra to Main Tier

Clicking on a %gra relation jumps through the %mor tier to the aligned main tier word. The %gra tier indexes into %mor positions (e.g., 3|2|OBJ means “word 3 depends on word 2”), so the extension follows the chain: %gra item -> %mor position -> main tier word.

This two-hop navigation is resolved transparently – you click once and land on the main tier word.

How It Works

The extension’s go-to-definition handler inspects the cursor position to determine what kind of element you clicked:

1. Speaker code – resolves to the @Participants header line via the speaker registry.
2. %mor item – resolves to the main tier word at the same alignment index.
3. %gra item – parses the source index from the source|head|relation triple, maps it through %mor alignment, and resolves to the main tier word.

All resolution uses the typed ChatFile AST produced by talkbank-parser, never raw text offsets.

Related Chapters

• Cross-Tier Alignment – hover tooltips showing the full alignment chain
• Dependency Graphs – visual representation of %gra relations
• Document Symbols – F2 rename and find-all-references for speaker codes
• Syntax Highlighting – visual distinction between tiers

Cross-Tier Alignment

Status: Current Last updated: 2026-04-16 22:24 EDT

CHAT transcripts are layered: a main tier line carries the spoken words, while dependent tiers (%mor, %gra, %pho, %sin) carry aligned annotations. The extension surfaces these alignment relationships through hover tooltips, document highlighting, and smart selection, so you never have to count positions manually.

Hover: Cross-Tier Alignment

Hover over any word or tier item to see how it aligns with elements on other tiers. The tooltip is computed from the parsed ChatFile model via talkbank-model’s positional alignment, not from string-position heuristics.

Hovering on a Main Tier Word

Place your cursor over a word on a main tier line (e.g., *CHI: I want cookie .). A tooltip appears showing every tier item aligned to that word:

Main Tier Word: "cookie"

- %mor tier
  POS: n (noun)
  Stem: cookie

- %gra tier
  3|2|OBJ -- OBJ -> want (word 2)

- %pho tier
  kuki

---
Alignment computed by talkbank-model

(SCREENSHOT: Hover popup on main tier word showing %mor, %gra, and %pho alignment) Capture this: hover the cursor over the word “cookie” on a *CHI: main tier line that has %mor, %gra, and %pho dependent tiers. The tooltip should show all three aligned items.

Hovering on a %mor Item

Hover over a morphological analysis item to see its full breakdown, including POS subcategory, stem, and any prefixes, suffixes, clitics, or translations:

Morphology Element: "pro:sub|I"

POS: pro:sub (subject pronoun)
Stem: I

<- Main tier: "I" (word 1)

- %gra tier
  1|2|SUBJ -- SUBJ -> want (word 2)

Compound morphology (n|+n|pine+n|apple), fusional items, and clitic chains are all displayed when present.

Hovering on a %gra Item

Hover over a grammatical relation to see the dependency it encodes, with back-links to both the %mor tier and the main tier:

Grammar Relation: "3|2|OBJ"

Source: cookie (word 3)
Head: want (word 2)
Relation: OBJ

<- %mor tier: n|cookie
<- Main tier: "cookie"

Other Dependent Tiers

Hovering also works on %pho (phonological transcription), %mod (model phonology), and %sin (gesture/sign language) tier items, showing their alignment back to the main tier word and, where applicable, the corresponding %mor item.

Header Hover

Hovering over @ headers (@Languages, @Participants, @ID, @Media, etc.) shows inline documentation describing the header’s purpose, syntax, and valid values. The @ID hover is especially detailed, displaying a field-by-field table explaining each of the 10 pipe-delimited fields (language, corpus, code, age, sex, group, SES, role, education, custom).

Bullet Hover

Hovering over a timing bullet (*NNN_NNN*) shows the formatted start time, end time, and duration of the segment in human-readable form (e.g., 01:23.456 - 01:25.789 (2.333s)). This makes it easy to inspect timing without mentally converting millisecond values.

Stale-baseline indicator

When you edit a document so heavily that the parser cannot produce a fresh ChatFile (typically because of an in-progress syntax error), alignment-consuming hover cards continue working from the last successful parse rather than going blank. In that state the hover appends a footer:

⚠ Stale baseline — alignment reflects the last successful parse.

The vocabulary — “stale baseline” — mirrors the internal ParseState::StaleBaseline identifier so the same term appears in server logs and source code. Once your edit parses cleanly the footer disappears on the next hover.

Document Highlighting: Visual Alignment

Click on any word or tier item and all aligned elements across tiers are highlighted simultaneously. This provides an instant visual map of how a single word flows through the annotation layers.

For example, clicking on the word “cookie” on the main tier highlights:

• “cookie” on the main tier (primary highlight, write access)
• “n|cookie” on the %mor tier (secondary highlight, read access)
• “3|2|OBJ” on the %gra tier (secondary highlight, read access)
• The corresponding item on %pho, %mod, or %sin if present

This works bidirectionally: clicking a %gra item highlights both its source word and head word on %mor, plus the corresponding main tier words.

(SCREENSHOT: Cross-tier highlighting with a word selected on the main tier) Capture this: click on a word like “cookie” on a main tier line that has %mor, %gra, and %pho tiers. All aligned items across tiers should show colored highlights.

Clitics and the %mor Chunk Sequence

English contractions (it's, I'll, can't), French elisions (l'homme), and similar multi-word tokens appear as a single orthographic word on the main tier but expand into multiple morphemes on the %mor tier. CHAT writes the expansion with a post-clitic marker ~. Each side of the ~ gets its own %gra relation. Understanding how the extension resolves alignments through clitic groups explains why hover, highlights, and the dependency graph all “just work” on real transcripts.

Worked example

Consider this line:

*CHI:   it's cookies .
%mor:   pron|it~aux|be noun|cookie .
%gra:   1|2|SUBJ 2|0|ROOT 3|2|OBJ 4|2|PUNCT

The %mor tier has two items — pron|it~aux|be and noun|cookie — but it has three non-terminator chunks: the pron|it main word, the aux|be post-clitic, and noun|cookie. Adding the terminator gives a total of four chunks. Every %gra relation aligns with exactly one chunk, not one item.

Key property: the post-clitic chunk and its host’s main chunk share the same main-tier word. Clicking either the SUBJ relation (1|2|SUBJ → chunk 0) or the ROOT relation (2|0|ROOT → chunk 1) highlights the same word it's on the main tier. The extension collapses chunks to their host %mor item before projecting through the main↔%mor alignment, which is keyed by item position.

Why this matters

%gra relation indices are 1-indexed positions in the %mor chunk sequence (main word, then each post-clitic, then the terminator), while the main↔%mor alignment is keyed by %mor item position. Every handler that projects a %gra relation onto the main tier collapses chunks to their host item via MorTier::item_index_of_chunk; without that step, any %gra click on a post-clitic silently lands on the next item after the clitic (usually the wrong word). The single primitive is used by:

• The %gra relation hover (word stem, head stem, dependents list).
• The %gra click → main-tier highlight handler.
• The dependency-graph DOT builder (each chunk gets its own node; edges connect chunk node IDs directly).
• The %gra → main-tier Go to Definition jump.

See the Developer: Architecture chapter for the model-side primitive (MorTier::chunks(), MorChunk<'a>) and the three distinct index spaces that the Rust types now enforce at the compiler level.

Smart Selection

Use VS Code’s Expand Selection command to expand through CHAT structural levels:

The expansion follows CHAT document structure:

1. Word – a single word or tier item
2. Tier content – all content on the current tier line
3. Full line – the entire tier line including the tier label
4. Utterance block – the main tier line plus all its dependent tiers
5. Entire file – the whole document

This mirrors VS Code’s standard selection expansion but is tuned for the CHAT document structure, making it easy to select exactly the scope you need for copying, deleting, or searching.

Related Chapters

• Dependency Graphs – visualize %gra tier relations as interactive arc diagrams
• Go to Definition – jump from a dependent tier item to its aligned main tier word
• Document Symbols – navigate the file by speakers and headers
• Syntax Highlighting – color coding for tiers and CHAT constructs
• Validation – alignment mismatch diagnostics

Dependency Graphs

Status: Current Last updated: 2026-04-16 22:25 EDT

The extension can render the grammatical dependency structure of any utterance as an interactive, color-coded graph. This turns the compact %gra tier notation (1|2|SUBJ 2|0|ROOT 3|2|OBJ) into a visual diagram where you can see at a glance how words relate syntactically.

Opening a Graph

1. Place your cursor on an utterance that has both %mor and %gra tiers.
2. Invoke the command using any of these methods:
   • Keyboard: Cmd+Shift+G (macOS) / Ctrl+Shift+G (Windows/Linux)
   • Context menu: right-click, then select Show Dependency Graph
   • Command Palette: Cmd+Shift+P and type TalkBank: Show Dependency Graph
3. A side panel opens with the rendered graph.

If the utterance lacks a %gra tier, the command reports a message and does nothing.

Graph Layout

Words are displayed as labeled boxes, arranged left-to-right in utterance order (matching the main tier). Colored arcs connect each word to its syntactic head, with the grammatical relation label on each arc. The ROOT relation connects to an invisible root node above the word row.

Relation Color Coding

(SCREENSHOT: Dependency graph for a complex utterance with colored arcs) Capture this: open a dependency graph for an utterance like *CHI: I want the big cookie . that has SUBJ, OBJ, DET, and MOD relations. The graph should show multiple colored arcs.

Toolbar Controls

The toolbar at the top of the graph panel provides zoom and export controls:

Singleton Panel

The graph panel reuses a single tab. Invoking the command on a different utterance updates the existing panel rather than opening a new one. This keeps the editor layout clean when you are stepping through multiple utterances.

Stale-baseline indicator

If you request a graph while the current document has parse errors that block a fresh reparse, the graph renders from the last successful parse rather than failing. In that state the DOT output gains a muted top-left label:

stale baseline

rendered in Courier 10pt, color #888888. The label is subordinate to the graph content (placement + font choice keep it out of the visual center) and disappears the next time you request a graph after a successful parse.

This is one of two surfaces that expose ParseState::StaleBaseline to users — the other is alignment-consuming hover cards (see Cross-Tier Alignment → Stale-baseline indicator). Go to Definition, highlights, and inlay hints stay silent in the same state because their single-position outputs are typically still correct, and a warning there would be noise.

How It Works

The rendering pipeline is entirely offline – no internet connection required.

1. Server side (chatter-lsp, in graph/): The LSP server receives the request via the talkbank/showDependencyGraph custom command. It extracts word labels from the %mor tier (graph/labels.rs), styles each %gra relation with a color (graph/edges.rs), and assembles a Graphviz DOT digraph (graph/builder.rs) with invisible ordering edges to preserve left-to-right word order.

2. Client side (graphPanel.ts): The extension receives the DOT string and passes it to the bundled @hpcc-js/wasm Graphviz renderer, which runs Graphviz entirely in WebAssembly. The rendered SVG is injected into a webview panel. The toolbar provides zoom, fit, and export controls over the SVG DOM.

Because the Graphviz engine is bundled as WASM (@hpcc-js/wasm), the entire pipeline works offline on any platform – no external Graphviz installation is needed.

Related Chapters

• Cross-Tier Alignment – hover and highlighting for %mor/%gra alignment
• Go to Definition – jump from %gra items to the main tier word
• CLAN Analysis Commands – further syntactic analysis via CLAN commands
• LSP Protocol – details of the talkbank/showDependencyGraph custom command

Speaker Filtering

Status: Current Last updated: 2026-03-30 13:40 EDT

In multi-party conversations, it is often useful to focus on a single speaker’s contributions. The speaker filtering feature creates a read-only filtered view of the transcript showing only the utterances from selected speakers.

How to Use

1. Open a .cha file.
2. Invoke the command using any of these methods:
   • Context menu: right-click in the editor, then select Filter by Speaker…
   • Command Palette: Cmd+Shift+P (macOS) / Ctrl+Shift+P (Windows/Linux), then type TalkBank: Filter by Speaker
3. A multi-select picker appears listing all speakers defined in the file’s @Participants header.
4. Select one or more speakers and confirm.
5. A filtered read-only document opens in a side-by-side panel.

What the Filtered View Shows

The filtered document includes:

• All file headers – @UTF8, @Begin, @Participants, @ID, @Languages, @Media, and all other @ headers are preserved so the document remains structurally valid.
• Selected speakers’ utterance blocks – each main tier line for the selected speakers, along with all their dependent tiers (%mor, %gra, %pho, etc.).
• No other speakers – utterance blocks from non-selected speakers are omitted entirely.

The filtered view is a virtual document – it does not modify the original file on disk. The language server still processes it, so you get full validation, highlighting, and hover information even in the filtered view.

Use Cases

• Focus on child speech: filter to CHI only to review a child’s productions without the surrounding adult turns.
• Examine interviewer questions: filter to INV to see just the interviewer’s prompts.
• Compare two speakers: select both CHI and MOT to see the mother-child dyad without other participants.
• Quick utterance review: combine with the Document Symbols outline to jump between the filtered speaker’s utterances.

Related Chapters

• Document Symbols – rename, find references, and code lens for speaker codes
• Scoped Find – search within specific tiers or speakers without creating a filtered view
• Cross-Tier Alignment – hover and highlighting work in the filtered view
• CLAN Analysis Commands – many CLAN commands accept --speaker flags for per-speaker analysis

Scoped Find

Status: Current Last updated: 2026-03-30 13:40 EDT

VS Code’s built-in Find (Cmd+F) searches the entire file as raw text. Scoped Find adds CHAT-aware filtering: search within specific tiers, specific speakers, or both. This replaces the tedious workflow of visually scanning tier lines or using fragile regex patterns that try to match CHAT structure.

How to Use

1. Open a .cha file.
2. Invoke the command using any of these methods:
   • Context menu: right-click in the editor, then select Find in Tier…
   • Command Palette: Cmd+Shift+P (macOS) / Ctrl+Shift+P (Windows/Linux), then type TalkBank: Find in Tier
3. A picker appears with two filter dimensions:
   • Tier type: main tier (*SPK:), %mor, %gra, %pho, %sin, %act, %cod, %com, %exp, or all tiers.
   • Speaker: optionally restrict the search to utterance blocks belonging to one or more speakers.
4. Enter your search query. Supports both plain text and regex (prefix with / for regex mode).
5. Results appear in a QuickPick list with clickable matches that navigate directly to the matching location in the file.

Search Modes

Plain Text

Type a word or phrase to find exact matches within the selected tier scope. The search is case-insensitive by default.

Example: searching for cookie in the %mor tier finds all morphological items containing “cookie” (e.g., n|cookie, n|cookie-PL).

Regex

Prefix your query with / to use regular expression matching.

Example: searching for /v\|want in the %mor tier finds all verb entries for “want” (matching v|want, v|want-PAST, etc.).

Use Cases

• Find a word in one speaker’s speech: select the main tier, filter to CHI, and search for cookie. This finds only the child’s productions of that word, ignoring adult uses.
• Search %mor for POS tags: select the %mor tier and search for adj| to find all adjective uses across the file.
• Find specific dependency relations: select the %gra tier and search for SUBJ to find all subject relations.
• Search annotations: select %cod or %act to search within coding or speech act tiers only.

How It Works

The scoped find uses the parsed ChatFile model to determine tier boundaries. It identifies which spans of the document belong to each tier type and each speaker, then searches raw text only within those spans. This means it never produces false matches from other tiers or headers – unlike a raw regex search that might accidentally match a %mor pattern appearing in a comment or header.

Results appear in a VS Code QuickPick list. Selecting a result navigates the cursor to the exact location of the match.

Comparison with Legacy CLAN

Legacy CLAN has no equivalent to scoped find. Users must use the generic find-and-replace dialog without any tier or speaker awareness, relying on manual inspection to filter irrelevant matches. The scoped find feature brings the semantic awareness of CLAN’s KWAL and COMBO search commands into the interactive editor experience.

Related Chapters

• Speaker Filtering – create a filtered view of selected speakers’ utterances
• Cross-Tier Alignment – hover and highlighting to explore tier relationships
• CLAN Analysis Commands – kwal and combo for batch search across files
• Keyboard Shortcuts – bind scoped find to a custom shortcut

Media Playback

Status: Current Last updated: 2026-03-30 13:40 EDT

Play audio or video files linked via @Media: headers directly inside VS Code. The extension sources timing segments from the language server’s alignment sidecar when available, falling back to direct bullet parsing (bulletParser.ts) when the LSP is unavailable.

Play at Cursor

Shortcut: Cmd+Shift+Enter (macOS) / Ctrl+Shift+Enter (Windows/Linux)

Plays the single bullet segment nearest to the cursor position. A media panel opens in the editor showing playback controls. The segment is determined by the timing bullet on or nearest to the current editor line — for example, if the cursor is on a *CHI: line containing go home ., playback covers the time range in that line’s bullet.

(SCREENSHOT: Media panel with speed slider and playback controls) Capture this: open a .cha file with media, press Cmd+Shift+Enter, and screenshot the media panel showing the speed slider, play/pause button, and time display.

Continuous Play

Shortcut: Cmd+Shift+/ (macOS) / Ctrl+Shift+/ (Windows/Linux)

Plays all segments from the cursor position through to the end of the file. The editor selection tracks the currently-playing utterance line, scrolling the document to keep the active line visible.

Segments play sequentially in document order (top to bottom), not sorted by media time. When one segment finishes, playback seeks to the next segment’s start time.

[VIDEO: 30s demo of continuous playback) Capture this: start continuous playback from the middle of a multi-speaker file. Show the editor cursor advancing through utterance lines as each segment plays.

Playback Speed

The media panel toolbar includes a speed slider ranging from 0.25x to 2x. Drag the slider to adjust playback speed — the current rate is shown next to the slider. This is particularly useful for slowing down fast speech during transcription or alignment review.

Rewind

Shortcut: F8

Rewinds playback by a configurable number of seconds (default: 2). The rewind amount is controlled by the talkbank.transcription.rewindSeconds setting:

{
  "talkbank.transcription.rewindSeconds": 3
}

The minimum value is 0.5 seconds. Rewind works during both single-segment and continuous playback.

Loop

Shortcut: F5

Toggles looping of the current segment. When enabled, the current bullet’s time range repeats automatically. The loop count is controlled by talkbank.walker.loopCount:

Playback State Machine

The following diagram shows the states a playback session transitions through:

stateDiagram-v2
    [*] --> Idle
    Idle --> Loading : Cmd+Shift+Enter\nor Cmd+Shift+/
    Loading --> Playing : media loaded,\nsegment resolved
    Loading --> Error : media not found\nor decode failure
    Playing --> Paused : pause button
    Paused --> Playing : resume button
    Playing --> SegmentEnd : currentTime >= seg.end
    SegmentEnd --> Playing : loop count > 0\n(replay same segment)
    SegmentEnd --> Seeking : next segment exists\n(continuous mode)
    SegmentEnd --> Idle : no more segments\nor single-segment mode
    Seeking --> Playing : seek complete
    Playing --> Rewinding : F8
    Rewinding --> Playing : seek complete\n(currentTime - rewindSeconds)
    Error --> [*]
    Idle --> [*]

The playback loop in mediaPanel.js polls currentTime every 100 ms (matching CLAN’s polling interval). When currentTime >= seg.end / 1000, the loop either advances to the next segment (continuous mode), replays (loop mode), or stops (single-segment mode).

Overlapping Bullets (Cross-Speaker Overlap)

CHAT allows cross-speaker overlapping bullets — two different speakers can have utterances whose time ranges overlap, as long as utterances are ordered by start time in the file (validation rule E701). Same-speaker self-overlap beyond 500 ms is prohibited by E704.

When the file contains overlapping bullets, playback works as follows:

• Play at Cursor plays exactly the clicked utterance’s time range. No other utterances are affected.

• Continuous Play plays each utterance in full, in document order. If speaker A’s utterance covers 0–3.5s and speaker B’s covers 2–2.5s, you hear A’s full range, then B’s full range. The overlapping audio region (2–2.5s) is heard twice — once as part of each speaker’s turn. This matches CLAN behavior and lets you hear each speaker’s complete utterance without truncation.

• Waveform overlays render each segment independently. Overlapping segments appear as stacked colored bars on the waveform (see Waveform Visualization).

This behavior is intentional. For transcripts with frequent backchannels and short overlapping speech (common in aphasia protocols, conversation analysis, and multi-party recordings), hearing each speaker’s complete turn in sequence is more useful than trying to play simultaneous audio streams.

Requirements

1. The .cha file must contain an @Media: header specifying the filename and type (audio or video). For example: @Media: interview, audio
2. The referenced media file must be findable by the extension’s media resolver — see Media Resolution for the full search order.
3. The media format must be playable by the browser engine embedded in VS Code (typically MP3, WAV, MP4, WebM).

Stop Playback

Use the Command Palette (Cmd+Shift+P) and search for TalkBank: Stop Playback to halt any active playback session.

See Also

• Waveform Visualization — visual audio waveform with segment overlays
• Walker Mode — step through utterances one at a time with auto-play
• Transcription Mode — create transcripts with F4 timing stamps
• Media Resolution — how the extension finds media files
• Keyboard Shortcuts — customize playback keybindings

Waveform Visualization

Status: Current Last updated: 2026-03-30 13:40 EDT

Visualize the audio waveform of the linked media file with timed utterance overlays. The waveform panel uses the Web Audio API to decode and render audio amplitude data, with colored segments marking each speaker’s bullet ranges.

Opening the Waveform

Shortcut: Cmd+Shift+W (macOS) / Ctrl+Shift+W (Windows/Linux)

1. Open a .cha file with an @Media: header
2. Press Cmd+Shift+W
3. A waveform panel opens showing the audio amplitude as a continuous waveform trace

The waveform panel is a singleton — pressing the shortcut again when the panel is already open brings it to focus rather than creating a duplicate.

(SCREENSHOT: Waveform panel with colored bullet segments) Capture this: open a multi-speaker .cha file with media, press Cmd+Shift+W, and screenshot the waveform panel showing colored segment overlays for different speakers. Ensure at least two speaker colors are visible.

Colored Segment Overlays

Each bullet segment is drawn as a colored overlay on the waveform. Speakers are assigned distinct colors, so you can visually distinguish which parts of the audio belong to which participant. Overlapping bullets from different speakers appear as stacked colored bars (see Playback — Overlapping Bullets).

Interaction

Click to Seek

Click anywhere on the waveform to seek to that time position. When you click:

• The media panel (if open) seeks to the clicked time
• The editor cursor moves to the utterance line nearest to that time
• If playback is active, it continues from the new position

This makes it easy to jump to a specific point in the audio by visual inspection of the waveform shape.

Coordination with Media Panel

The waveform and media panels are fully coordinated:

• Playing a segment highlights it in the waveform
• Seeking in the waveform updates the media panel position
• The currently-playing segment is visually distinguished (brighter overlay or outline) so you can track playback progress

Zoom

The waveform panel provides several zoom controls:

Pointer-Centered Zoom

Mouse wheel zooming is centered on the pointer position. This means the point under your cursor stays fixed as you zoom in, allowing precise navigation to a specific region of the audio. This is particularly useful when examining a short segment within a long recording.

Scroll

When zoomed in beyond the panel width, the waveform scrolls horizontally:

• Manual scroll: Use the horizontal scrollbar or shift+scroll wheel
• Auto-scroll during playback: When playback is active, the view automatically scrolls to keep the current segment visible. The scroll follows the playback position so you never lose sight of what is currently playing.

Typical Workflow

1. Open a .cha file and press Cmd+Shift+W to open the waveform
2. Press Cmd+Shift+Enter to open the media panel
3. Click on the waveform to jump to a region of interest
4. Zoom in with the scroll wheel for fine-grained inspection
5. Use Walker Mode (Alt+Down / Alt+Up) to step through utterances while watching the waveform highlight each segment

See Also

• Media Playback — audio/video playback controls and behavior
• Walker Mode — step through utterances with synchronized waveform
• Transcription Mode — use the waveform for visual reference while transcribing
• Media Resolution — how the extension finds the audio file

Walker Mode

Status: Current Last updated: 2026-03-30 13:40 EDT

Step through utterances one segment at a time, playing each segment’s audio automatically. Walker mode is the primary tool for reviewing transcript alignment — verifying that each utterance’s timing bullet matches the actual speech in the audio.

Keyboard Shortcuts

Each step moves the editor cursor to the corresponding utterance line. If media is available and auto-play is enabled, the segment’s audio plays immediately.

[VIDEO: 30s demo of walker stepping through utterances) Capture this: open a .cha file with media and the waveform panel. Press Alt+Down repeatedly, showing the cursor advancing through utterance lines while each segment’s audio plays and the waveform highlights the active segment.

How It Works

1. The walker maintains a pointer to the current utterance index in the document
2. Pressing Alt+Down advances the pointer and moves the editor cursor to the next main tier line (a line starting with *SPK:)
3. If talkbank.walker.autoPlay is enabled, the utterance’s bullet segment plays automatically
4. The waveform panel (if open) highlights the active segment
5. After playback completes, the walker waits for the next keypress (or pauses for pauseSeconds in continuous mode)

Configuration

All walker settings live under the talkbank.walker namespace:

Example Configuration

To set up the walker for careful alignment review — playing each segment twice with a 1-second pause between utterances:

{
  "talkbank.walker.autoPlay": true,
  "talkbank.walker.loopCount": 2,
  "talkbank.walker.pauseSeconds": 1
}

To step through utterances without audio (text-only review):

{
  "talkbank.walker.autoPlay": false
}

Document Change Reset

The walker resets its position when the active document changes. Switching to a different .cha file or a non-CHAT document clears the walker state. When you return to the original file, stepping resumes from wherever the cursor is positioned.

Tips

• Combine with the waveform panel (Cmd+Shift+W) for visual feedback — you can see the highlighted segment on the waveform as you step through utterances.

• Use with Review Mode for systematic post-alignment review. The Post-Alignment Review workflow describes the full process of verifying that timing bullets match the actual speech.

• Adjust loop count for difficult segments. When reviewing a transcript with fast or unclear speech, set loopCount to 2 or 3 so each segment replays automatically before advancing.

• Use walkLength for focused review. If you only need to review the next 10 utterances, set walkLength to 10. The walker stops after that many steps rather than continuing to the end of the file.

See Also

• Media Playback — playback controls, speed, rewind, and loop
• Waveform Visualization — visual waveform with segment highlighting
• Transcription Mode — create new transcripts (walker is for reviewing existing ones)
• Post-Alignment Review — full review workflow
• Settings Reference — all extension settings

Transcription Mode

Status: Current Last updated: 2026-03-30 13:40 EDT

Create new CHAT transcripts from audio by typing utterance text while the audio plays, then stamping timing bullets at utterance boundaries. Transcription mode turns VS Code into a transcription workstation — the CLAN equivalent of Transcribe mode.

Prerequisites

The .cha file must contain an @Media: header pointing to an audio or video file. The media file must be findable by the extension’s resolver (see Media Resolution). A minimal starting file looks like:

@UTF8
@Begin
@Languages: eng
@Participants: CHI Target_Child
@Media: interview, audio
*CHI:
@End

Starting Transcription Mode

1. Open the .cha file
2. Open the Command Palette (Cmd+Shift+P)
3. Search for TalkBank: Start Transcription Mode
4. Audio begins playing from the start of the file

The waveform panel opens automatically to provide visual reference.

Workflow

The core transcription loop is:

1. Listen to the audio as it plays
2. Type the speaker’s utterance text on the current *SPK: line
3. Press F4 to stamp a timing bullet and advance

When you press F4:

• A timing bullet is inserted at the end of the current utterance line, marking the time span from the previous stamp (or file start) to the current playback position: go home .
• The cursor automatically advances to a new *SPK: line, ready for the next utterance
• The speaker code on the new line uses the configured default speaker

4. Repeat until the recording is complete
5. Open the Command Palette and search for TalkBank: Stop Transcription Mode

[VIDEO: 60s demo of transcription workflow with F4 stamping) Capture this: start transcription mode on a short audio file. Type 3-4 utterances, pressing F4 after each one. Show the timing bullets being inserted and new speaker lines being created. Include the waveform panel in the recording.

Key Commands During Transcription

Configuration

Example Configuration

For an investigator transcribing an adult interview:

{
  "talkbank.transcription.defaultSpeaker": "INV",
  "talkbank.transcription.rewindSeconds": 3
}

Foot-Pedal and Custom Keybindings

USB foot pedals are commonly used in transcription to control playback without taking hands off the keyboard. To remap transcription keys for a foot pedal:

1. Open the Command Palette (Cmd+Shift+P)
2. Search for TalkBank: Configure Transcription Keybindings
3. The VS Code keybindings editor opens, pre-filtered to TalkBank transcription commands
4. Click the pencil icon next to any command to assign a new keybinding

This is the recommended way to map foot-pedal buttons to F4 (stamp), F8 (rewind), and F5 (loop). Most USB foot pedals present as standard keyboard devices and can be mapped to any VS Code keybinding.

See Keyboard Shortcuts for more on customizing keybindings.

Tips

• Use the waveform panel for visual reference. The waveform shows where you are in the recording and helps you identify utterance boundaries by looking at silence gaps between speech regions.

• Rewind generously. The default 2-second rewind (F8) lets you re-hear the end of an utterance. For fast or overlapping speech, increase rewindSeconds to 3 or 4.

• Slow down playback using the speed slider on the media panel. Dropping to 0.5x or 0.75x gives you more time to type without pausing.

• Edit timing after transcription. Transcription mode creates an initial pass. Use Walker Mode afterward to review each segment and adjust timing bullets as needed.

See Also

• Media Playback — playback controls, speed, and rewind
• Waveform Visualization — visual waveform for boundary identification
• Walker Mode — review and adjust alignment after transcription
• Media Resolution — how the extension finds media files
• Transcription Workflow — end-to-end transcription process
• Keyboard Shortcuts — customize keybindings

Media Resolution

Status: Current Last updated: 2026-03-30 13:40 EDT

This chapter explains how the extension locates media files referenced by CHAT transcripts. Understanding the resolution process helps when media playback fails or when organizing corpus files on disk.

The @Media Header

Every .cha file that uses media features must include an @Media: header specifying the base filename and the media type:

@Media: interview, audio
@Media: session03, video

The header provides two pieces of information:

1. Base filename — the name without extension (e.g., interview)
2. Media type — either audio or video, which determines the file extensions the resolver tries

For audio, the resolver looks for: .mp3, .wav, .m4a, .ogg, .flac

For video, the resolver looks for: .mp4, .mov, .avi, .webm, .mkv

Resolution Search Order

When you trigger playback or open the waveform, the extension’s media resolver (mediaResolver.ts) searches for the media file in the following order:

1. Same directory as the .cha file
2. media/ subdirectory — a sibling media/ folder next to the .cha file
3. Configured media roots — additional search paths from extension settings

At each location, the resolver tries every applicable file extension for the declared media type. The first match wins.

Example

Given a file at /corpus/English/Brown/Adam/adam01.cha with header @Media: adam01, audio, the resolver searches:

/corpus/English/Brown/Adam/adam01.mp3
/corpus/English/Brown/Adam/adam01.wav
/corpus/English/Brown/Adam/adam01.m4a
  ...
/corpus/English/Brown/Adam/media/adam01.mp3
/corpus/English/Brown/Adam/media/adam01.wav
/corpus/English/Brown/Adam/media/adam01.m4a
  ...

Document Links

@Media file references are rendered as clickable links in the editor. If the resolver finds the media file, clicking the link opens it directly in the system’s default media player (or in VS Code’s built-in preview for supported formats).

This provides a quick way to verify that the media file exists and is accessible without starting playback mode.

Picture Display

The extension can display elicitation pictures (Cookie Theft, picture descriptions, etc.) alongside the transcript — the CLAN equivalent of PictController.

Opening a Picture

1. Open a .cha file
2. Right-click in the editor and select TalkBank: Media then Show Elicitation Picture, or use the Command Palette

How Pictures Are Found

The extension searches for associated images using three strategies, in order:

1. %pic: tier references — if the document contains dependent tier lines like %pic: "image002.jpg", those filenames are used directly

2. Same-name image files — image files matching the .cha file’s base name (e.g., adam01.cha matches adam01.jpg, adam01.png)

3. Directory contents — any image files in the same directory as the .cha file

Multi-Image Picker

If multiple candidate images are found, a picker dialog appears listing all matches. Select the desired image to display it.

Display

The selected picture opens in a side panel and scales to fit the available space. You can arrange the picture panel alongside the transcript and waveform for a complete annotation workspace.

Troubleshooting

When media playback or the waveform panel fails to open, check the following:

1. Verify the @Media Header

The .cha file must contain a valid @Media: line. Check for:

• Missing header entirely — add @Media: filename, audio (or video)
• Typos in the filename — the base name must match the actual media file
• Wrong media type — audio vs video must match the actual file format

2. Check That the File Exists

Verify that the media file is present in one of the resolver’s search locations (same directory or media/ subdirectory). The filename must match the @Media base name exactly (case-sensitive on macOS and Linux).

3. Check the File Extension

The media file must have a recognized extension for its type. An .mp3 file declared as video in the @Media header will not be found — the resolver only tries video extensions (.mp4, .mov, etc.) for video-typed headers.

4. Check the Output Panel

Open the Output panel in VS Code (Cmd+Shift+U) and select the TalkBank channel. The extension logs media resolution attempts and failures here, showing exactly which paths were searched and why resolution failed.

5. Large or Unsupported Formats

VS Code’s embedded browser engine (Chromium) has limits on media format support. If the file exists but playback fails, the format may not be supported. Common working formats are MP3, WAV, and MP4 (H.264). Uncommon codecs (e.g., FLAC in some configurations, AVI with legacy codecs) may not play.

See Also

• Media Playback — playback controls and behavior
• Waveform Visualization — visual audio waveform
• Media Troubleshooting — extended troubleshooting guide

Running CLAN Commands

Status: Current Last updated: 2026-04-16 16:55 EDT

The TalkBank extension includes 33 CLAN analysis commands that run directly inside VS Code. No external CLAN binary is needed – all analysis runs inside the Rust language server.

Pipeline

Every analysis command — single-file or directory — follows the same pipeline:

flowchart LR
    u["User<br/>(context menu / Command Palette)"]
    qp{{"QuickPick<br/>(33 commands)"}}
    needsIn{"Command<br/>needs extra<br/>input?"}
    inbox["Input dialog<br/>(text / file picker)"]
    cancel[["Cancel → exit"]]
    scope{"Single file<br/>or directory?"}
    walk["Recursive walk<br/>resolve_files()"]
    dispatch["LSP talkbank/analyze<br/>(typed request)"]
    runner["AnalysisRunner<br/>(clan-core crate)"]
    result["JSON result<br/>(typed Response)"]
    panel["AnalysisPanel<br/>webview"]

    u --> qp
    qp --> needsIn
    needsIn -->|"yes (6 commands)"| inbox
    needsIn -->|"no"| scope
    inbox -->|"user provides"| scope
    inbox -->|"user cancels"| cancel
    scope -->|"single file"| dispatch
    scope -->|"directory"| walk
    walk --> dispatch
    dispatch --> runner
    runner --> result
    result --> panel
    panel -->|"Export CSV"| csv(["CSV file<br/>on disk"])

The six commands that prompt for extra input are listed below under Commands that prompt for additional input. Directory analysis reuses the single-file dispatch per file and aggregates results in the panel.

Running an analysis on a single file

1. Open a .cha file in the editor.
2. Right-click anywhere in the editor and choose Run CLAN Analysis…, or open the Command Palette (Cmd+Shift+P / Ctrl+Shift+P) and select TalkBank: Run CLAN Analysis.
3. A QuickPick list appears with all 33 commands. Each entry shows a short description of what the command computes.
4. Select a command. If the command requires additional input (see below), a follow-up dialog appears.
5. Results open in a styled side panel with stat cards, tables, and proportional bar charts.

(SCREENSHOT: Context menu showing “Run CLAN Analysis…” and “Run KidEval…” entries) Capture this: right-click in an open .cha file to show the full context menu with CLAN analysis options

Running an analysis on a directory

To analyze all .cha files in a directory at once:

1. In the Explorer sidebar, right-click a folder.
2. Choose Run CLAN Analysis on Directory….
3. Select an analysis command from the picker.
4. The language server walks the directory tree recursively and runs the command on every .cha file it finds.
5. Results from all files are aggregated in a single analysis panel.

This is equivalent to CLAN’s batch-mode directory analysis and is useful for computing corpus-wide statistics such as total vocabulary or mean MLU across participants.

The analysis panel

Results display in a WebviewPanel with several presentation elements:

• Stat cards at the top showing key numeric results (e.g., MLU value, type-token ratio, total word count) with color-coded indicators.
• Tables with sortable columns for detailed breakdowns (e.g., word frequencies, speaker-level statistics).
• Proportional bar charts for visual comparison of values across speakers or categories.

(SCREENSHOT: Analysis panel showing FREQ results with stat cards at top and frequency table below) Capture this: run FREQ on a multi-speaker .cha file, showing the stat cards and word frequency table

Exporting results as CSV

Click the Export CSV button in the analysis panel toolbar. A save dialog appears where you choose the output location and filename. The CSV file includes:

• All stat card values as key-value rows.
• All visible tables with their column headers.

The CSV format is compatible with Excel, Google Sheets, R (read.csv()), and Python (pandas.read_csv()). For longitudinal studies, export results from multiple files and combine them in a spreadsheet.

Commands that prompt for additional input

Six commands require information beyond the .cha file itself. After selecting one of these commands, a follow-up dialog appears:

For mortable, script, and rely, the file picker dialog opens to the current workspace folder by default. You can navigate to any location on disk.

Tips

• Run the same command on multiple files by using the directory analysis option. This is faster than running the command file by file.
• Compare speakers within a single file: most commands (FREQ, MLU, MLT, VOCD) break results out per speaker automatically.
• Combine with speaker filtering: use Speaker Filtering first to focus on specific participants, then run an analysis.
• Save CSV exports with consistent naming conventions for longitudinal data collection (e.g., CHI_freq_session01.csv).

Next steps

• Profiling Commands – MLU, MLT, VOCD, DSS, IPSyn
• Frequency & Distribution – FREQ, WDLEN, MAXWD, and more
• Assessment Tools – KidEval, Eval, Eval-D
• Command Reference – all 33 commands in one table

Profiling Commands

Status: Current Last updated: 2026-03-30 13:40 EDT

Profiling commands measure structural properties of a speaker’s language: how long their utterances are, how diverse their vocabulary is, and how grammatically complex their sentences are. These are the most commonly used measures in child language research and clinical assessment.

All profiling commands are available through the standard analysis workflow described in Running CLAN Commands.

MLU – Mean Length of Utterance

What it measures: The average number of morphemes per utterance. MLU is the single most widely used index of grammatical development in young children (Brown, 1973).

How it works: The command counts morphemes on the %mor tier for each utterance, then computes the mean across all utterances for each speaker. If the file has no %mor tier, it falls back to counting words on the main tier.

When to use it:

• Tracking grammatical development in children aged 1;6 to 5;0
• Screening for language delay (compare against age norms)
• As a baseline measure in treatment studies

Results include:

• MLU value per speaker
• Total morphemes and total utterances
• Standard deviation

Tip: MLU is most meaningful for children in Brown’s Stages I–V (roughly MLU 1.0 to 4.5). Beyond MLU 4.5, it becomes less sensitive to development. For older children, consider DSS or IPSyn instead.

MLT – Mean Length of Turn

What it measures: The average number of utterances per conversational turn, along with words per turn. A “turn” is a sequence of consecutive utterances by the same speaker, bounded by utterances from other speakers.

How it works: The command identifies turn boundaries by detecting speaker changes, counts the utterances and words in each turn, and computes the mean.

When to use it:

• Measuring conversational participation and verbosity
• Comparing turn-taking patterns between child and adult speakers
• Assessing pragmatic language skills

Results include:

• Mean utterances per turn
• Mean words per turn
• Total turns per speaker

VOCD – Vocabulary Diversity

What it measures: The D statistic, a measure of vocabulary diversity that is more robust than a simple type-token ratio (TTR). TTR is heavily influenced by sample size – longer samples always produce lower TTR. VOCD corrects for this by computing D from random subsamples of varying sizes.

How it works: The command draws random subsamples of 35 to 50 tokens from the transcript, computes TTR for each subsample size, and fits a mathematical curve to the TTR-by-sample-size function. The D parameter of that curve is the vocabulary diversity score. Higher D indicates greater diversity.

When to use it:

• Comparing lexical diversity across speakers or sessions with different sample lengths
• Research on vocabulary development
• Any context where simple TTR would be misleading due to unequal sample sizes

Results include:

• D statistic per speaker
• Type count and token count
• Type-token ratio (for reference, but prefer D)

Tip: VOCD requires a minimum of about 50 tokens to produce a reliable estimate. Very short transcripts will produce unstable D values.

DSS – Developmental Sentence Scoring

What it measures: Grammatical complexity of children’s sentences, scored against a weighted system of eight grammatical categories: indefinite pronouns, personal pronouns, main verbs, secondary verbs, negatives, conjunctions, interrogative reversals, and wh-questions. Each sentence receives a total score; the mean across sentences is the DSS score.

How it works: The command applies scoring rules from the DSS rule files in the clan-info library. Only complete sentences with a subject and predicate are scored. Incomplete utterances, single-word responses, and imitations are excluded.

Language support: DSS rules are available for:

When to use it:

• Assessing grammatical maturity in children aged 2;0 to 6;11
• Identifying children with specific language impairment
• Measuring treatment outcomes for grammar intervention

Results include:

• DSS score per speaker
• Per-sentence scores
• Category-level breakdown (pronouns, verbs, negatives, etc.)
• Number of sentences scored vs. excluded

Tip: DSS is designed for English-speaking children. For non-English transcripts, consider language-specific profiling tools or use MLU as a general measure.

IPSyn – Index of Productive Syntax

What it measures: The range of syntactic and morphological structures a child produces, based on a checklist of 56 (or 100) language forms across four categories: noun phrases, verb phrases, questions/negation, and sentence structure.

How it works: The command searches the first 100 utterances for evidence of each structure on the checklist. Each structure earns 0 points (not found), 1 point (found once), or 2 points (found in two different utterances). The maximum score is 112 (56-item version) or 200 (100-item version).

Rule files:

When to use it:

• Assessing syntactic development in children aged 1;6 to 4;6
• Complementing MLU with a more detailed structural profile
• Research on syntactic bootstrapping and acquisition order

Results include:

• Total IPSyn score per speaker
• Subscale scores (noun phrase, verb phrase, question/negation, sentence)
• Per-item breakdown showing which structures were found

Tip: IPSyn samples only the first 100 utterances. If your transcript is longer, only the first 100 contribute to the score. This is by design – the measure assesses productive range, not frequency.

Choosing the right profiling command

Next steps

• Frequency & Distribution – word counts and distributions
• Assessment Tools – compare profiling results against normative databases with KidEval
• Command Reference – all 33 commands

Frequency & Distribution

Status: Current Last updated: 2026-03-30 13:40 EDT

Frequency and distribution commands count words, morphemes, and phonological segments in various ways. They answer questions like “what words appear most often?”, “how long are the words?”, and “which parts of speech dominate the transcript?”.

All commands in this chapter are available through the standard analysis workflow described in Running CLAN Commands.

FREQ – Word Frequency

What it does: Counts every word in the transcript and reports the frequency of each unique word form, organized by speaker.

Results include:

• Total word tokens (running words)
• Total word types (unique forms)
• Type-token ratio (TTR)
• Frequency table: each word with its count, sorted by frequency

FREQ is the most general-purpose counting command. It is the starting point for any lexical analysis.

Tip: Use the CSV export to load frequency lists into a spreadsheet for further analysis. Sort by frequency to find the most common words, or alphabetically to check for spelling variants.

FREQPOS – Frequency by Part of Speech

What it does: Counts word frequency broken down by part-of-speech categories from the %mor (morphology) tier.

Requires: A %mor tier. Files without morphological coding will produce empty results.

Results include:

• Frequency table with columns for word, POS tag, and count
• Summary counts per POS category (nouns, verbs, adjectives, etc.)

When to use it: When you need to know not just which words appear, but what grammatical roles they fill. For example, a child might have a large vocabulary but use very few verbs – FREQPOS reveals this.

WDLEN – Word Length Distribution

What it does: Reports the distribution of word lengths (measured in characters) across the transcript.

Results include:

• Histogram of word lengths (how many 1-character words, 2-character words, etc.)
• Mean word length per speaker
• Longest and shortest words

When to use it: For phonological or literacy research where word length is a variable of interest. Comparing word length distributions across sessions can reveal changes in lexical sophistication.

WDSIZE – Vocabulary Size

What it does: Reports the cumulative vocabulary size – the number of unique word types – as the transcript progresses. This produces a vocabulary growth curve.

Results include:

• Total vocabulary size per speaker
• Character length statistics from %mor stems

When to use it: For studying vocabulary growth over the course of a session or across sessions. The growth curve shows whether a speaker is still introducing new words or has plateaued.

MAXWD – Longest Words

What it does: Finds and lists the longest words produced by each speaker, measured in characters.

Results include:

• Longest words per speaker, with their lengths
• Context (the utterance in which each word appeared)

When to use it: As a quick check on lexical sophistication, or to spot transcription errors (unusually long “words” may be coding mistakes or missing spaces).

PHONFREQ – Phonological Frequency

What it does: Counts the frequency of phonological segments on the %pho (phonology) tier.

Requires: A %pho tier with IPA transcription.

Results include:

• Frequency of each phoneme
• Consonant vs. vowel breakdown
• Position-in-word analysis (initial, medial, final)

When to use it: For phonological research, speech sound disorder assessment, or studying sound acquisition patterns.

MODREP – Model and Replica

What it does: Analyzes imitation patterns by comparing a child’s utterance (the replica) to the preceding adult utterance (the model). Reports what was repeated exactly, what was modified, and what was added or omitted.

When to use it: For studying imitation and language learning strategies. High exact-imitation rates in older children may indicate echolalia; selective imitation patterns reveal what structures the child is actively acquiring.

COOCCUR – Co-occurrence

What it does: Counts how often pairs of words appear together in the same utterance.

Results include:

• Co-occurrence matrix or list of word pairs with counts
• Statistical association measures

When to use it: For studying lexical associations, collocations, and formulaic language. Useful in discourse analysis and vocabulary research.

DIST – Distribution

What it does: Analyzes the distribution and dispersion of specific words or codes across the transcript, showing where they cluster and where they are absent.

Results include:

• Distribution profile across utterance ranges
• Clustering and dispersion statistics

When to use it: For studying whether certain words or behaviors are evenly distributed or concentrated in particular portions of the interaction.

UNIQ – Unique Forms

What it does: Identifies repeated and unique utterances in the transcript. Finds exact duplicates and near-duplicates.

Results include:

• List of repeated utterances with their frequencies
• Unique utterances (appearing only once)

When to use it: For identifying stereotyped or formulaic language, echolalia, or transcription artifacts (accidental duplicate lines).

Working with frequency results

CSV export for spreadsheet analysis

All frequency commands support CSV export via the Export CSV button in the analysis panel. This is the recommended workflow for quantitative analysis:

1. Run FREQ (or another frequency command) on a file or directory.
2. Click Export CSV in the panel toolbar.
3. Open the CSV in your analysis tool of choice.

Combining with other commands

Frequency results are most informative when paired with profiling measures:

• Run FREQ to get raw counts, then VOCD to get a sample-size-corrected diversity measure. See Profiling Commands.
• Run FREQPOS to see POS distributions, then MLU to relate vocabulary composition to utterance length.
• Use FREQ on a directory to get corpus-wide word lists for normative comparison or stimuli selection.

Next steps

• Profiling Commands – MLU, MLT, VOCD, DSS, IPSyn
• Assessment Tools – KidEval, Eval, Eval-D
• Command Reference – all 33 commands in one table

Assessment Tools

Status: Current Last updated: 2026-03-30 16:35 EDT

The assessment tools — KidEval, Eval, and Eval-D — compare a speaker’s language measures against normative databases. Instead of reporting raw numbers (MLU = 3.2), they tell you how that number compares to age-matched peers (z-score = -1.5, below average).

These tools have dedicated commands and panels, separate from the general CLAN analysis picker. They are the primary workflow for clinical assessment and research benchmarking.

KidEval — Child Language Assessment

KidEval computes a battery of language measures (MLU, MLT, VOCD, DSS, IPSyn, and others) and compares each against normative data from age-matched, typically developing children.

How to run KidEval

1. Open a .cha file containing child language data.
2. Right-click in the editor and choose Run KidEval, or use the Command Palette: TalkBank: Run KidEval.
3. The KidEval panel opens with characteristic selection controls.

Step 1: Select language and activity

The panel presents two cascading dropdowns:

• Language — choose the child’s language (English NA, English UK, French, Japanese, Spanish, Chinese, Dutch, Cantonese)
• Activity Type — choose the data collection context (Narrative or Toyplay). The activity dropdown filters automatically to show only the activities available for the selected language.

If the CHAT file has @Languages and @Types headers, the panel auto-detects the correct language and activity — no manual selection needed.

Below the dropdowns, an availability grid shows all language × activity combinations at a glance. Click any cell to select it:

              Narrative    Toyplay
English (NA)   ● 375       ● 3503
English (UK)   ● 540       ● 3459
French         ● 399       ● 627
Japanese       —            ● 167
Spanish        ● 106       ● 936
Chinese        ● 392       ● 1196
Dutch          —            —       *
Cantonese      —            —       *

● = available with sample count
— = not available
* = general database (not activity-specific)

When a language + activity is selected, a matching database card appears showing the database filename, sample count, and metadata.

The activity type matters: narrative tasks (story retelling, picture description) and toy play sessions elicit different language patterns. Use the database that matches your data collection protocol.

Step 2: Filter by age and gender

After selecting language and activity, you can optionally narrow the comparison group:

• Gender — filter to Male, Female, or All
• Age range — enter minimum and maximum age in months to compare against children within a specific developmental window

Leave these fields blank to compare against the full normative sample.

(SCREENSHOT: KidEval panel showing language/activity dropdowns, availability grid with a cell selected, and the matching database card) Capture this: open the KidEval panel on an English narrative .cha file. The language should be auto-selected to “English (NA)”, activity to “Narrative”, and the grid should highlight the eng/narrative cell. Show the matching database card below.

Available KidEval databases

Step 3: View results

Click Run to compute the analysis. Results appear in three sections:

Stat cards at the top of the panel show the key measures with color-coded z-score indicators:

• Green: within 1 standard deviation of the mean (typical range)
• Yellow: between 1 and 2 standard deviations (borderline)
• Red: more than 2 standard deviations from the mean (atypical)

(SCREENSHOT: KidEval results showing stat cards with z-scores and comparison table) Capture this: KidEval results after clicking “Compare with Database” for an English narrative file, showing colored stat cards (green/red z-scores) and the detailed comparison table with Measure, Child, Norm Mean, Norm SD, z-score, N columns.

Comparison table showing side-by-side values:

Per-measure breakdown with additional detail for each metric.

What each measure means

For detailed explanations of each measure, see Profiling Commands.

Eval – General Language Assessment

Eval uses the same panel interface as KidEval but with databases designed for adult or general language assessment rather than child development norms.

How to run Eval

1. Open a .cha file in the editor.
2. Right-click and choose Run Eval, or use the Command Palette: TalkBank: Run Eval.
3. Select a database, optionally filter, and click Run.

Available Eval databases

Eval databases contain normative data from healthy adult speakers, useful as a baseline for comparing clinical populations.

Eval-D – Dementia Assessment

Eval-D is a specialized assessment mode for evaluating language in dementia contexts. It uses normative databases from DementiaBank.

How to run Eval-D

1. Open a .cha file in the editor.
2. Right-click and choose Run Eval-D (Dementia), or use the Command Palette: TalkBank: Run Eval-D (Dementia).
3. Select a database and click Run.

Available Eval-D databases

Eval-D results use the same panel layout as KidEval and Eval: stat cards with z-scores, comparison table, and per-measure breakdown. The normative groups include control participants and multiple dementia subtypes, allowing clinicians to see where a patient falls relative to each group.

Exporting assessment results

All three assessment tools support CSV export:

1. After results are displayed, click Export CSV in the panel toolbar.
2. Choose a save location.
3. The CSV includes all stat card values, z-scores, and the full comparison table.

This is useful for:

• Importing results into a clinical report template
• Longitudinal tracking (export at each visit, combine in a spreadsheet)
• Research data collection across multiple participants

Choosing the right assessment tool

Next steps

• Profiling Commands – understand what MLU, DSS, and IPSyn measure in detail
• Running CLAN Commands – how to run analysis commands in general
• Command Reference – all 33 commands

Command Reference

Status: Current Last updated: 2026-03-30 13:40 EDT

Complete reference for all 33 CLAN analysis commands available in the TalkBank extension. Run any command via right-click Run CLAN Analysis… or the Command Palette. See Running CLAN Commands for the general workflow.

All commands

Commands by category

Frequency (11 commands)

Word counting, vocabulary, and distribution analysis. See Frequency & Distribution for detailed descriptions.

• freq – word frequency counts
• freqpos – frequency by part of speech
• wdlen – word length distribution
• wdsize – vocabulary size
• maxwd – longest words
• phonfreq – phonological segment frequency
• modrep – model and replica (imitation)
• cooccur – word co-occurrence
• corelex – core vocabulary
• dist – word distribution
• uniq – unique/repeated utterances

Profiling (8 commands)

Structural and developmental measures of language. See Profiling Commands for detailed descriptions.

• mlu – Mean Length of Utterance
• mlt – Mean Length of Turn
• vocd – Vocabulary Diversity (D)
• dss – Developmental Sentence Scoring
• ipsyn – Index of Productive Syntax
• chip – child/parent interaction profile
• complexity – syntactic complexity ratio
• sugar – SUGAR narrative sampling

Assessment (3 commands)

Normative comparison against reference databases. See Assessment Tools for detailed descriptions.

• kideval – child language assessment
• eval – general language assessment
• evald – dementia assessment

Search (2 commands)

Finding utterances by keyword or pattern.

• kwal – keyword in context
• combo – Boolean keyword search (AND/OR)

Coding (3 commands)

Working with coding tiers (%cod).

• codes – code frequency counts
• chains – code chains and sequences
• keymap – keyword contingency mapping

Morphology (1 command)

Custom morphological tabulation.

• mortable – morpheme frequency table from a script

Other (5 commands)

Specialized tools for specific tasks.

• flucalc – fluency/disfluency calculation
• rely – inter-rater reliability
• script – template comparison
• timedur – timing duration
• trnfix – tier mismatch detection

Library files used by commands

Several commands use rule files from the clan-info/lib/ directory. These files ship with the extension and are discovered automatically by the language server.

Next steps

• Running CLAN Commands – how to run commands and export results
• Profiling Commands – detailed guide to MLU, VOCD, DSS, IPSyn
• Frequency & Distribution – detailed guide to FREQ, WDLEN, and more
• Assessment Tools – KidEval, Eval, Eval-D normative comparison

Rating Utterances

Status: Current Last updated: 2026-03-30 13:40 EDT

Rating values

What “Good” means

The bullet points to the audio location where the words in the utterance are actually spoken. A few milliseconds of imprecision is fine — “Good” means “I can hear the right words when I play from this bullet.”

What “Early” means

The bullet plays audio from before the utterance starts. You hear silence, the previous speaker’s words, or background noise before the current utterance’s words begin.

This is the most common issue with boundary-averaged bullets — the algorithm split the difference between two overlapping utterances, but split it in the wrong place.

What “Late” means

The bullet plays audio that starts partway through the utterance — you miss the first word or two. The speaker is already talking when playback begins.

What “Wrong” means

The bullet is wildly off — you hear a completely different part of the conversation, a different speaker, or content that doesn’t match the text at all. This typically happens when the alignment algorithm lost sync.

Adding notes

After pressing a rating key, the %xrev tier updates automatically. If you want to add a note, edit the line directly in the editor:

%xrev:	[early] starts about 200ms before speech, consistent with other INV lines

Notes don’t affect the rating — they’re free text after the marker. The harvesting script collects them for manual analysis.

Useful note patterns

• Timing offset: “early by ~300ms” or “late by about half a second”
• Speaker pattern: “all INV backchannels seem early”
• Overlap pattern: “this is where two speakers talk over each other”
• Algorithm feedback: “LIS shouldn’t have stripped this — timing was correct”
• Correction hint: “should start at roughly 12300ms, not 12095ms”

Interactive Bullet Correction

Status: Current Last updated: 2026-03-30 13:40 EDT

Status: This feature is planned but not yet implemented. This chapter documents the design for when it ships.

Beyond rating utterances as good/early/late/wrong, the extension will support interactive correction — clicking on the waveform to set the correct bullet boundary.

Why correction is more valuable than rating

• Rating “early” tells us the direction but not the magnitude
• A corrected bullet tells us “was 12095ms, should be 12295ms” — a precise measurement of how far off the algorithm was
• Corrected bullets become ground truth for tuning algorithm thresholds
• The %xalign tier records both values: machine=12095_12335 corrected=12295_12335

Planned workflow

1. During Review Mode, navigate to a flagged utterance
2. Click on the waveform where speech actually begins — the bullet start time updates
3. Shift-click to set the end time
4. Preview the adjusted region to verify
5. Accept (Enter) — the correction is saved and %xrev set to [corrected]
6. Revert (Escape) — restore the original bullet

What gets recorded

When you correct a bullet, the %xalign tier is updated with both the machine’s original timing and your correction:

%xalign:	boundary_averaged overlap=155ms machine=12095_12335 corrected=12295_12335 delta_start=+200ms
%xrev:	[corrected]

The delta_start=+200ms field tells us exactly how much the algorithm was off, in which direction. Aggregated across many corrections, this data reveals systematic biases that we can fix in the algorithm.

Stamping untimed utterances

For utterances where align stripped timing entirely (LIS removal), you can stamp a new bullet from scratch using the same F4 mechanism as Transcription Mode:

1. Play the audio and listen for the utterance
2. Press F4 at the start to stamp the begin time
3. Press F4 again at the end to stamp the end time
4. The %xrev tier is set to [stamped]

%xalign:	lis_removal same_speaker_non_monotonic machine=none stamped=11940_27820
%xrev:	[stamped] timing was actually correct

Harvesting Review Results

Status: Current Last updated: 2026-03-30 13:40 EDT

After reviewers have rated utterances in aligned CHAT files, the ratings need to be collected for analysis. The %xrev and %xalign tiers are part of the CHAT files, so harvesting is just parsing.

Using the harvesting script

python3 scripts/harvest_reviews.py path/to/reviewed/files/ --output reviews.csv

This scans all .cha files in the directory, extracts %xrev and %xalign tiers, and produces a CSV with one row per reviewed utterance:

What to do with the results

Aggregate quality metrics

# How many rated good vs early vs late vs wrong?
cut -d, -f5 reviews.csv | sort | uniq -c | sort -rn

Per-strategy breakdown

# Are boundary-averaged bullets rated differently than gap-filled?
awk -F, '{print $4, $5}' reviews.csv | sort | uniq -c | sort -rn

Displacement analysis (for corrected bullets)

# Average displacement for corrected bullets
awk -F, '$5=="corrected" && $11!="" {sum+=$11; n++} END {print sum/n "ms avg"}' reviews.csv

Pattern detection

Look for systematic issues in the notes column:

grep -i 'backchannel\|overlap\|speaker' reviews.csv

Comparing conditions

To compare raw align vs --bullet-repair vs FIXBULLETS, review the same files under each condition and harvest separately:

python3 scripts/harvest_reviews.py raw-output/ --output raw.csv
python3 scripts/harvest_reviews.py repaired-output/ --output repaired.csv
python3 scripts/harvest_reviews.py fixbullets-output/ --output fixbullets.csv

Then compare the %good rate across conditions.

Codes Files (.cut)

Status: Current Last updated: 2026-03-30 13:40 EDT

Codes files (.cut) define the hierarchical coding scheme used by Coder Mode. Each file describes the valid codes for a particular research task – speech act annotation, error coding, conversational strategy labeling, and so on.

File Format

A .cut file is a plain text file with the following structure:

1. Header lines – metadata and CLAN configuration options (lines starting with #, \, @, or the magic first line)
2. Target tier declaration – a %xxx: line specifying which dependent tier receives the codes (e.g., %spa:, %err:, %cod:)
3. Code hierarchy – indented lines defining the valid codes and their tree structure

Header Section

The file begins with CLAN-specific metadata that the VS Code extension ignores:

lXs Special Text file saves all fonts LxS
@Font:	CAfont:13:0
@UTF8
# coder mode commands:
# ESC-e  = toggle between coders and text editing modes
# ...
\ +b50 +d +l1 +s1 +fCAfont

Lines starting with # are comments. The \ line contains CLAN coder options (auto-save interval, font selection, reorder behavior). These are specific to CLAN’s modal editor and have no effect in the VS Code implementation.

Target Tier

A line like %spa: or %err: declares the dependent tier that will receive the codes. When you select a code in the QuickPick, it is inserted as this tier type on the current utterance.

Code Hierarchy

Codes are organized in an indented tree. Each level of indentation represents a level in the hierarchy. Codes typically follow the pattern $CATEGORY:subcategory:

%spa:
 " $POS
  :QUE
   :NV
   :VE
  :ANS
   :NV
   :VE
  :COM
   :NV
   :VE
  :ACK
   :VE

In this example:

• $POS is a top-level category (positive speech acts)
• :QUE, :ANS, :COM, :ACK are subcategories under $POS
• :NV and :VE are leaf codes under each subcategory (non-verbal and verbal)

The " prefix marks speaker scope boundaries (e.g., " $MOT means the following codes apply when coding the mother’s utterances).

Example: codes-basic.cut

The simplest standard code file, codes-basic.cut, defines a basic speech act scheme for mother-child interaction:

%spa:
 " $MOT
  :POS
   :Que
   :Res
  :NEG
 " $CHI

This creates a hierarchy where:

• Mother ($MOT) utterances can be coded as positive (:POS) with subtypes question (:Que) or response (:Res), or as negative (:NEG)
• Child ($CHI) utterances use the same top-level scheme

Example: codeserr.cut

The error coding file defines categories for lexical errors on the %err tier:

%err:
 " $LEX
  :INC     incomplete
  :UNC     uncertain
  :ACC     accented
  :ADD     added
  :LOS     lost/omitted
  :HAP     haplology
  :BLE     blend
  :EX1     example type 1
  :EX2     example type 2
  :SH1     shift type 1
  :SH2     shift type 2

Standard Code Files

The following .cut files are distributed with TalkBank in clan-info/lib/coder/:

Creating Custom .cut Files

To create a custom coding scheme:

1. Start from a template. Copy one of the standard .cut files and modify it, or start from scratch with the minimal structure below.

2. Minimal structure:

   @UTF8
   %cod:
    " $TOP
     :sub1
     :sub2
      :leaf1
      :leaf2
   

3. Choose your target tier. Use %cod: for general-purpose coding, %spa: for speech acts, %err: for error coding, or any valid CHAT dependent tier prefix.

4. Define the hierarchy. Use single-space indentation for each level. Category codes start with $, subcodes start with :.

5. Keep it flat enough to navigate. The VS Code QuickPick displays the hierarchy with indentation. Deep hierarchies (more than 4 levels) become difficult to scan in the picker.

Tips for Organizing Code Hierarchies

• Use meaningful category prefixes. $PRA for pragmatic functions, $ERR for errors, $NIA for conversational acts. The prefix makes codes identifiable when reading coded tiers in the transcript.

• Balance breadth and depth. A flat list of 50 codes is hard to navigate; a deep tree with 6 levels is also hard. Aim for 3-4 levels with 5-10 items per level.

• Document your codes. Add # comment lines in the .cut file explaining what each category means. Both CLAN and the VS Code extension ignore comment lines.

• Use speaker scoping sparingly. The " prefix scopes codes to specific speakers. This is useful when different participants have different valid codes (e.g., adult vs. child speech acts), but adds complexity.

• Test with a small file first. Open the .cut file in Coder Mode with a short transcript to verify the hierarchy displays correctly in the QuickPick before coding a large corpus.

Related Chapters

• Coder Mode Overview – what coding is and why to use it
• Coding Workflow – step-by-step usage in VS Code

Coding Workflow

Status: Current Last updated: 2026-04-16 16:45 EDT

This chapter walks through the complete Coder Mode workflow in VS Code, from starting a coding session to finishing and reviewing the results.

Prerequisites

Before starting:

• A .cha transcript file open in the editor
• A .cut codes file defining your coding scheme (see Codes Files)

Starting Coder Mode

1. Open a .cha file in the editor
2. Open the Command Palette (Cmd+Shift+P on macOS, Ctrl+Shift+P on Windows/Linux)
3. Type “Start Coder Mode” and select TalkBank: Start Coder Mode
4. A file picker appears – navigate to and select your .cut codes file
5. The extension parses the .cut file and loads the code hierarchy

Alternatively, right-click in the editor and select TalkBank: Navigation then Start Coder Mode.

Once started, the cursor jumps to the first utterance that does not already have a coded tier. A status indicator confirms that Coder Mode is active.

(SCREENSHOT: Command Palette showing “Start Coder Mode” with the file picker for .cut files) Capture this: Open Command Palette, type “Start Coder”, show the QuickPick file selector with .cut files listed

Inserting Codes

With Coder Mode active, press Cmd+Shift+C (macOS) or Ctrl+Shift+C (Windows/Linux) to open the code picker.

The QuickPick displays all codes from the .cut file as a flat list with indentation to show the hierarchy:

  $POS
    :QUE
      :NV
      :VE
    :ANS
      :NV
      :VE
    :COM

Type to filter the list using VS Code’s fuzzy matching. Select a code and press Enter.

The selected code is inserted as a dependent tier immediately after the current utterance:

*CHI:	I want the red one .
%cod:	$PRA:request

The picker placeholder shows progress information (e.g., “12 of 45 utterances coded”) so you can track how much work remains.

(SCREENSHOT: QuickPick code picker showing hierarchical codes with progress indicator) Capture this: During active Coder Mode, press Cmd+Shift+C to show the picker with indented codes and the “X of Y coded” placeholder text

Stepping Through Utterances

After inserting a code, press Cmd+Enter (macOS) or Ctrl+Enter (Windows/Linux) to advance to the next uncoded utterance. The editor scrolls to center the utterance, and the cursor is positioned on the main tier line.

The stepping logic is backed by the LSP via the talkbank/getUtterances command. It identifies utterances that already have a coded tier and skips them, so you only visit utterances that need coding.

If all utterances are coded, the stepping command displays a notification that coding is complete.

Keyboard Shortcuts

These shortcuts are active only when Coder Mode is running (the talkbank.coderActive context is set):