CRC SEAQ: System for the Evaluation of Audio Quality
CRC-TimeSync: Audio Waveform Synchronization

CRC-SEAQ

Audio quality can be measured either subjectively, with human subjects who compare audio stimuli in listening tests and generate opinion scores, or objectively, with computations performed on the audio stimuli being tested. In both cases, the goal is to obtain a number that reliably represents the perceived audio quality.

CRC-SEAQ (System for the Evaluation of Audio Quality) is a Windows 95/98/NT4/2000/XP software package for the subjective and/or objective evaluation of the quality of audio signals, including speech. With its intuitive graphical user interfaces, CRC-SEAQ provides an environment and a comprehensive set of tools for users who wish to accurately measure audio quality while developing or evaluating audio technologies, products and equipment. The system was developed by researchers at the Communications Research Centre, in Ottawa, Canada who have been involved for many years in the development of subjective and objective methods for audio quality measurements.

CRC-SEAQ can be configured to include a Subjective Test Module and/or an Objective Test Module. CRC-TimeSync, a companion program for time synchronizing audio files, can also be included as a module in CRC-SEAQ or licensed separately.

CRC-SEAQ brochure

HTML | PDF 804 kb

You will need Adobe Acrobat to view this brochure. If this brochure is not accessible to you, please contact louis.thibault@crc.gc.ca for assistance.

Subjective Test Module

Current Version: v2.03.5

The Subjective Test Module is an audio playback system with a series of control buttons and features (triple-stimulus A-B-C or multi-stimulus presentation, switching, zooming, looping and scoring) provided in three easy-to-use graphical user interfaces (GUI). The triple-stimulus GUI is designed to conduct ITU-R Recommendation BS.1116 compliant subjective tests and is particularly well suited for detecting and rating signals with small audible impairments. The multi-stimulus GUI comes in two versions: a multi-button and a multi-slider. It is ideal when signals (up to 12) with medium and large impairments need to be compared and rated reliably against a reference, such as in the MUSHRA test method defined in ITU-R Recommendation BS. 1534. The audio sequences to be compared are read from files and played via any audio card with Windows MME support. The Subjective test Module supports a variety of audio signals and file formats, sampling frequencies and resolutions.

The Subjective Test Module is available in a 2-channel version, for mono and stereo signals, or a multichannel version which supports up to 16 audio channels.

The Subjective Test Module can be used for:

• Formal and informal listening tests
• Evaluating different implementations of audio processing equipment such as audio and speech codecs, 3D audio systems, etc.
• Status or reality checks during audio system development
• Perception and psychoacoustics research
• Demonstrations involving the comparison of different audio stimuli

Key Features:

• Three intuitive graphical user interfaces (user selectable)
• Compliant with ITU-R Recommendations for testing small, medium and large audible impairments.
• Seamless switching between signals during playback
• Versatile zooming on any portion of the audio sequences under test
• Automatic repeat (looping) of audio sequences
• Audio sequences easily grouped into listening sessions
• Built-in scoring window. Test scores readily imported into spreadsheets or other analysis software
• Standard and user-definable rating scales
• Compatible with virtually any audio card that provides Windows MME support
• Mono, 2-channel and multichannel audio sequences
• Supports 8, 16, 20, 24, and 32 bits/sample, and normalized floating point
• Supports sampling rates of 8, 11.025, 22.05, 32, 44.1, 48, 88.2 and 96 kHz
• Supports Intel and Motorola byte ordering for audio samples
• Compatible with Microsoft WAV and headerless PCM files

Objective Test Module

Current Version: v1.5

The Objective Test Module is an "electronic ear" which performs an objective measurement of perceived audio quality using a computerized model of the human ear. This model, known as PEAQ, was developed jointly by experts from eight leading research laboratories in the world, including the CRC, and is an international standard (ITU-R Recommendation BS.1387). The model processes the two audio signals to be compared (namely the original reference signal and an impaired version to be evaluated) and calculates instantaneously a quality rating similar to the mean rating that would be obtained if a formal subjective test were conducted. The basic (FFT-based) version of the ITU-R PEAQ model is implemented in the Objective Test Module.

The measuring instrument was calibrated using data from eight subjective listening tests conducted according to ITU-R Recommendation BS.1116. Much of the audio data in those tests were obtained from digital audio codecs. As a result, the measurement method is considered most reliable for evaluating codec quality. Other types of distortions may require re-calibration of the instrument.

The Objective Test Module is useful for:

• Evaluating different implementations of audio processing equipment
• Testing equipment or circuits before they are put into service
• Automated monitoring of the quality of an audio transmission in service
• Identifying the type and implementation of a particular audio codec
• Characterizing the performance of a coding algorithm during development
• Optimizing the cost and performance of a transmission network under given constraints

The Objective Test Module has three modes of operation:

• On-line Mode: This mode is used for automated real-time audio quality monitoring. The software monitors an incoming audio signal in order to recognize and capture a pre-defined reference test signal. Once captured, the impaired test signal is automatically synchronized with the unimpaired version stored on hard disk, and an objective quality measure is done.
• Off-line Mode: The two signals to be compared (i.e. original reference signal and impaired version) are stored as files on the user's PC. Both files can be automatically synchronized before an objective quality measure is calculated.
• Play and Measure Mode: This mode simplifies the task of measuring the quality of external audio devices. A reference signal is looped through the device under test and recorded back onto a PC. Once the recording is stopped the software automatically synchronizes the impaired signal with the reference signal before performing an audio quality assessment.

Key features:

• Three modes of operation: On-line, Off-line, Play and Measure
• Audio files may be specified interactively or via a batch file
• Analysis window duration and overlap is user-selectable
• Up to 12 model output variables may be displayed
• Interactive zoom and measurement of any portion of the audio sequences under test
• Automatic temporal alignment of audio sequences under test
• Audio files playable via the system audio card
• Supports mono and 2-channel audio sequences
• Supports 16, 20, 24, and 32 bits/sample, and normalized floating point
• Supports Intel and Motorola byte ordering for audio samples
• Supports sampling rates of 44.1 and 48 KHz
• Compatible with Microsoft WAV and headerless PCM files
• DOS command line interaction also included

Observe the variations in objective quality over time.

---

CRC-TimeSync

Current Version: v2.05

Audio synchronization is an issue any time two or more audio signals need to be compared. For instance, when measuring audio quality objectively, a reference audio signal is compared to an impaired version. In both cases it is critical that the two audio signals be synchronized in time. The presence of delays between audio signals can result in distorted quality measurements.

CRC-TimeSync is a Windows 95/98/NT4/2000/XP application that offers a simple solution to the task of audio file comparison and synchronization. The layout and controls of the graphical user interface provide the means to quickly establish sample accuracy between source files.

CRC-TimeSync also includes tandeming feature which greatly simplifies the task of recording audio materials when looped through external audio devices and codecs. The user needs only to specify the reference file and the number of generations desired. This can be used to measure, for example, the effect of multiple generations of audio encoding and decoding.

CRC-TimeSync is available as a standalone program or integrated with CRC-SEAQ (System for the Evaluation of Audio Quality). Files can be shared between the two programs, which together create a powerful audio testing software suite.

CRC-TimeSync brochure

HTML | PDF 358 kb

You will need Adobe Acrobat to view this brochure. If this brochure is not accessible to you, please contact louis.thibault@crc.gc.ca for assistance.

Key features:

• Record and play back audio files via the system audio card
• Manual or automatic time alignment of audio files
• Flexible zoom controls to view one or two files simultaneously
• Remove unwanted segments from the beginning or end of audio files
• Drag and drop for quickly loading audio files
• Seamless integration with CRC-SEAQ
• Supports CRC-SEAQ session files
• Batch syncronization with user-definable offsets
• View the correlation between audio files
• Audio-tandeming (requires a full duplex audio adapter)
• Parse or join multiple audio files
• Compatible with virtually any audio card that provides Windows MME support
• Supports mono and 2-channel audio sequences
• Supports 8, 16, 20, 24, and 32 bits/sample, and normalized floating point
• Supports sampling rates of 8, 11.025, 22.05, 32, 44.1, 48, 88.2 and 96 kHz
• Supports Intel and Motorola byte ordering for audio samples
• Compatible with Microsoft WAV and headerless PCM files

Hardware Requirements

Pentium Pro or Pentium II CPU 400MHz or better, Minimum 128MB RAM, 5200rpm or faster hard drive (IDE or SCSI), Microsoft compatible three-button mouse, Audio adapter with Windows MME support, Windows 95/98/NT4/2000/XP.

For more information:

Mr. Louis Thibault
Manager, Advanced Audio Systems
Communications Research Centre
3701 Carling Ave., P.O. Box 11490, Station H
Ottawa, Ontario, Canada K2H 8S2
Tel:+1 613 990-4349
Fax:+1 613 993-9950
E-mail: louis.thibault@crc.gc.ca