Canmetry
Open workbench
Back to blog

How to Import a Large ASC File into MATLAB

canMessageImport runs out of memory on large ASC logs. Four ways to get decoded CAN signals into MATLAB: trim first, read in chunks, use BLF or decode outside.

.asc.dbcMATLAB

You have a Vector .asc log with CAN messages and timestamps, a DBC that describes the signals, and MATLAB. The obvious route is canMessageImport followed by canSignalTimetable. On a long recording it stalls or fails with an out-of-memory error. This guide explains why, then shows four ways around it, from the quickest to the most flexible.

Why canMessageImport runs out of memory

Every frame becomes a row

canMessageImport(file, "Vector", db) reads the whole ASC file and returns one timetable with a row per frame: timestamp, ID, channel, data bytes, flags and, with a database attached, the decoded signals for that message. Nothing is filtered on the way in. Frames from buses you never look at take the same memory as the ones you need.

Decoding adds a second copy

canSignalTimetable(msgs, "VehicleStatus") then builds a new timetable per message from the rows above. For a while you hold the raw messages and the decoded signals at the same time, which is usually the point where MATLAB gives up.

How big a large ASC file is

ASC is plain text. A classic CAN frame takes about 50 bytes as a line, and a busy vehicle bus carries a few thousand frames per second. One hour on three channels is easily 1–2 GB of text and tens of millions of rows. The fix is the same in every option below: do not let MATLAB see frames you will not decode.

Option 1: Make the ASC file smaller first

This is the fastest fix when you know which minutes and which messages matter.

Trim to a time window and the IDs you need

The ASC splitter streams the file in your browser, so there is no size limit. It lists every CAN ID per channel with its frame count and draws the line density over the recording. Select the time window, untick the IDs you do not need and download a smaller .asc. Lines are copied exactly as recorded and the header is kept, so canMessageImport reads the result like the original.

Split it and import the parts one by one

If you need the whole recording, split it by duration instead and decode each part in a loop. Append the decoded values to a MAT file with matfile, so memory only ever holds one part:

db    = canDatabase("vehicle.dbc");
parts = dir("drive_part*.asc");

out = matfile("drive_speed.mat", "Writable", true);
out.time  = zeros(0, 1);
out.speed = zeros(0, 1);
n = 0;

for k = 1:numel(parts)
    msgs = canMessageImport(parts(k).name, "Vector", db);
    sig  = canSignalTimetable(msgs, "VehicleStatus");
    t = seconds(sig.Time);
    out.time(n+1:n+numel(t), 1)  = t;
    out.speed(n+1:n+numel(t), 1) = sig.VehicleSpeed;
    n = n + numel(t);
    clear msgs sig
end

Keep the original timestamps when you split (the default), so the appended values stay on one time axis.

Option 2: Read the ASC in chunks in MATLAB

If the file has to stay on a machine where you only have MATLAB, filter it while you read.

Keep only the frames you need

textscan can read a fixed number of lines at a time. Pick the frame ID out of each line with a regular expression, write the lines you keep to a new file, and import that file:

keep = ["1A0" "2B4"];          % IDs exactly as written in the log
fin  = fopen("drive.asc", "r");
fout = fopen("drive_small.asc", "w");
first = true;

while ~feof(fin)
    c = textscan(fin, "%s", 200000, "Delimiter", "\n", "Whitespace", "");
    lines = string(c{1});
    tok = regexp(lines, "^\s*\d+\.\d+\s+\d+\s+(\w+)\s+(Rx|Tx)", "tokens", "once");
    isFrame = ~cellfun(@isempty, tok);
    if first                    % copy the header that comes before the first frame
        fprintf(fout, "%s\n", lines(1:find(isFrame, 1) - 1));
        first = false;
    end
    hit = false(size(lines));
    hit(isFrame) = ismember(cellfun(@(t) string(t{1}), tok(isFrame)), keep);
    fprintf(fout, "%s\n", lines(hit));
end
fclose(fin); fclose(fout);

msgs = canMessageImport("drive_small.asc", "Vector", canDatabase("vehicle.dbc"));

Memory stays at one block of 200,000 lines. Expect a few minutes per gigabyte. The pattern above matches classic CAN lines only, so extend it if your log has CANFD rows.

Write results as you go

For very long logs, combine this with the matfile loop from Option 1: filter one block, decode it, append the signals and move on. A MAT file created by matfile is version 7.3, so it can grow past 2 GB.

Option 3: Convert to BLF and read only what you need

BLF is Vector's compressed binary log. It is several times smaller than ASC, and blfread can filter while it reads.

Convert the log

In CANoe or CANalyzer, use Logging File Conversion. Without a Vector licence, python-can does the same in a few lines:

import can

reader = can.ASCReader("drive.asc")
with can.Logger("drive.blf") as writer:
    for msg in reader:
        writer.on_message_received(msg)

Read with filters

blfread takes a channel, a database, an ID filter and a time range, so only matching frames reach memory:

db   = canDatabase("vehicle.dbc");
msgs = blfread("drive.blf", 1, "Database", db, ...
               "CANStandardFilter", hex2dec(["1A0" "2B4"]), ...
               "TimeRange", [600 900]);
sig  = canSignalTimetable(msgs, "VehicleStatus");

If your loggers already write MF4, keep MF4. It stores signals by channel group, so MATLAB can read just the signals and time range you ask for.

Option 4: Decode outside MATLAB and load only signals

The smallest thing you can hand MATLAB is the decoded signals themselves: one time vector and one value vector per signal, without the raw frames.

With Python

python-can reads an ASC file line by line, and cantools decodes each frame with your DBC. Only the signals you keep are held in memory:

import can, cantools, numpy as np
from scipy.io import savemat

db = cantools.database.load_file("vehicle.dbc")
wanted = {db.get_message_by_name(n).frame_id for n in ["VehicleStatus", "BrakeStatus"]}
cols = {}
for msg in can.ASCReader("drive.asc"):
    if msg.is_error_frame or msg.arbitration_id not in wanted:
        continue
    for name, value in db.decode_message(msg.arbitration_id, msg.data, decode_choices=False).items():
        t, v = cols.setdefault(name, ([], []))
        t.append(msg.timestamp)
        v.append(value)

savemat("drive_signals.mat", {n: {"time": np.array(t), "value": np.array(v)} for n, (t, v) in cols.items()})

In MATLAB, s = load("drive_signals.mat") gives one struct per signal, ready to plot.

Without code

The ASC to MAT converter does the same in your browser. Link a DBC to each channel, and it writes one struct per signal named Message_Signal, with time, value, unit, message and channel fields. load opens it without the Vehicle Network Toolbox. It takes files up to 150 MiB, five at a time, so for a bigger log split it by size first with the ASC splitter (for example 140 MB per part) and convert the parts in one batch.

Which option fits your log

Your situationBest option
You know the time window and messagesOption 1: trim, then import
You need the whole log in MATLABOption 1: split and import in a loop, or Option 4
Only MATLAB is allowed on the machineOption 2: chunked read
You have CANoe or many logs to archiveOption 3: convert to BLF
You only need decoded signalsOption 4: Python or the ASC to MAT converter

Summary

canMessageImport is fine for short logs, but it loads every frame before you can decode anything. For a large ASC file, reduce what MATLAB has to hold: trim or split the file first, filter it while reading, switch to a binary format that supports filtered reads, or decode outside MATLAB and load only the signals. Most people get there fastest with the first option: cut the log down in the ASC splitter, then import it as usual.

.blf.mf4.asc

Open a log. See the signal.

Open a BLF, MF4 or ASC file in your browser. Nothing is uploaded.

Open workbench

Keep reading