Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Log Parsing tool available

This site may earn commission on affiliate links.

scott451

KWH-PWR#1349Sprt,S Sig#96
Apr 2, 2009
258
3
Palo Alto
I’ve decided to release my tesla log parsing tool. It’s very basic. It will convert your tesla log file into an excel csv file. There are a lot of interesting values in the log like speed, current, torque, state of charge, etc. There are also a lot of values that are not decoded. I am releasing this tool to enlist your help to decode the remaining values. You can access the parser at the following website. [EDIT 11-15-2010. Website is off line. Please use attached zip file.]


You will still need to do some data crunching in excel to produce useful plots. As an example I added efficiency as ESS_I * ESS_V / (Line_I * Line_V) to the data collected in the log. Attached is an example plot created in excel from the output of my parser.

attachment.php?attachmentid=672&d=1272728135.gif


-Scott

Note: the temp data in the PERM_Daily is wrong. Also the chargeV2 record for 2010 is not supported. (I still need to collect the CAN bus data off a 2010 to figure this one out... )
 

Attachments

  • 24A charge on 4-10-2010.GIF
    24A charge on 4-10-2010.GIF
    33.5 KB · Views: 3,261
Last edited:
well done, Scott, thank you.
It worked for my 2010 EU signature.
After download, Excel said "file could not be fully loaded", but that might be because it is more than 65k lines?
Now I sit here with -a lot- of data. As I am not an Excel expert, I will have to find someone who will make "nice curves" out of that data. What I could read in the plain text is a lot of error messages like "Motor Sensor 1 (or 2) overTemp warning. Torque limited." which might reflect my driving style ;-).
I also could clarify some charging errror messages I had in the VDS "Charging Error check power source" which in the logfiles much more clearly says "Line underVoltage warning".
But what I do have extremely often, nearly every day in a dozen is an "isolation resistance warning".
Everyone else has this "isolation resistance warning"? It is not in the VDS nor in the speedo, so I think it might not be of that influence ... hopefully ...

best
Martin
 
well done, Scott, thank you.
It worked for my 2010 EU signature.
After download, Excel said "file could not be fully loaded", but that might be because it is more than 65k lines?
...
I also could clarify some charging errror messages I had in the VDS "Charging Error check power source" which in the logfiles much more clearly says "Line underVoltage warning".

The "file could not be fully loaded" is an Excel 2004 and earlier problem. Excel 2007 doesn't have this limitation. To get around it, look in the spread sheet and find the "utime" that is nearest the event you want to track. Copy that utime into the website "start time" field and re upload the file. The new csv file will start at the new utime. Additonally you can shorten the csv file in wordpad first and then load it into excel.

Once it is in excel, to get nice curves, data->sort on Column A and then select the data you are interested in and click the chart wizzard.

The error messages in the log file, will be slightly different than the ones you see on the VDS. The reason is that I use the "verbose error text" not the standard errors. The verbose errors are more descriptive. Collecting the error messages for the parser is a tedious process of analizing the data on the CAN bus and hand typing into my code. I've just about finished some simulation hardware, so this should make it somewhat easier, rather than having to collect the data from my car...

I should have the 2010 V2 charge data worked out in a few weeks. I need to bring my CAN bus stuff over to a 2010 owner and collect the data, but I've been too swamped at my real job...

-Scott
 
Last edited:
But what I do have extremely often, nearly every day in a dozen is an "isolation resistance warning".
Everyone else has this "isolation resistance warning"? It is not in the VDS nor in the speedo, so I think it might not be of that influence ... hopefully ...
I saw this once on the VDS. It was when I was running on a race track. I had gotten a number of the motor overtemp warnings that you mention, and once the isolation resistence one, too. I never saw it before or since.

Maybe what we should take away from this is that your everyday driving is like what I do on the track. :)
 
Scott,

Thanks for making your log analysis available to other owners.

The log files contain a bunch of information including your VIN, your driving second-by-second, your charging, and the GPS coordinates of everywhere you charge, which presumably includes your home location.

Although there's apparently no GPS data included with the driving records, I suspect that there is enough data that driving routes could be reconstructed by comparing driving patterns (speed, distance and stopping times) with stop lights and traffic flow data in your local area, thus potentially revealing a lot of personal information from those logs.

What's your data retention and privacy policy for the data that owners upload to your web server?
 
Scott,

Thanks for making your log analysis available to other owners.

The log files contain a bunch of information including your VIN, your driving second-by-second, your charging, and the GPS coordinates of everywhere you charge, which presumably includes your home location.
Not exactly. The log contains the places where you "limited" the charge. the log does not contain the location of every place you charge. the GPS data is ignored by my parser. The time of charge is not included in the GPS location so it is difficult to correlate to the charge record.

Although there's apparently no GPS data included with the driving records, I suspect that there is enough data that driving routes could be reconstructed by comparing driving patterns (speed, distance and stopping times) with stop lights and traffic flow data in your local area, thus potentially revealing a lot of personal information from those logs.
good luck :) the log does not contain distance travailed per second but speed sampled every second. It also doesn't contain the heading. So you don't know when the GPS location was stored, you don't know how much distance you've travailed in a second, and you don't know which direction you are going. Pretty tough to get any useful data. (e,g where I work, the route I took, etc)

What's your data retention and privacy policy for the data that owners upload to your web server?
Log files are used to improve the tool (e.g add more error messages). Parsed data is created and then deleted. uploaded log files are deleted regularly.


-Scott
 
Last edited:
Not exactly. The log contains the places where you "limited" the charge. the log does not contain the location of every place you charge. the GPS data is ignored by my parser. The time of charge is not included in the GPS location so it is difficult to correlate to the charge record.

I limit charging at home, so my home coordinates are in the log file. I know others do the same.

good luck :) the log does not contain distance travailed per second but speed sampled every second. It also doesn't contain the heading. So you don't know when the GPS location was stored, you don't know how much distance you've travailed in a second, and you don't know which direction you are going. Pretty tough to get any useful data. (e,g where I work, the route I took, etc)

Using speed * time = distance on the once-per-second records yields a reasonable estimate of distance traveled, with maybe a 5% to 10% error. That can be improved using the once-per-ten-minute records that contain the odometer to the 1/10th mile and interpolating. I haven't done the work to see how much accuracy can be obtained, but I can't dismiss that good accuracy is possible.

You can definitely reconstruct information like this: drove 10.2 miles at low speeds with several stops (city driving), drove 26.3 miles at 75 mph on average (got on the freeway at the exit which is 10.2 miles from home, then got off at the only exit 26.3 miles from the first), then more figuring things out by distances between stop lights. I'm not saying you could always figure things out exactly, but I'm sure someone who wanted to could reconstruct a lot of information.

I'm sure you're not doing this with the data that owners submit, but it is a privacy issue that I think owners should be aware of when they turn their log files over to Tesla, or anyone else.

Log files are used to improve the tool (e.g add more error messages). Parsed data is created and then deleted. uploaded log files are deleted regularly

So you're not keeping any identifiable data for more than a short period of time? That's great to know. I'd recommend stating that on your upload page.

Despite asking the annoying questions, I think it's great you're sharing your program with the community. Thanks for taking the time to answer my post.
 
Not exactly. The log contains the places where you "limited" the charge. the log does not contain the location of every place you charge. the GPS data is ignored by my parser. The time of charge is not included in the GPS location so it is difficult to correlate to the charge record.

Also. The latest version of the [2008, 2010, and sport] firmware encrypts the GPS charging location, vms and user config files, so they can't be read by anyone but tesla.
 
Last edited:
log reader parser Version _v12 online

I've updated my log parsing tool. The new version includes the following features:

Correct parsing of ChargeV2 records from 2010 Roadsters (thx to Richard for alowing me to log his 2010)
Added brick numbers to min temp, max temp, and min v, max V fields.
Added brake and regen bits to drive1s records (from tomsax PM)
Added support for parsing older hidden records cause by wraped logs.
Merged temp and perm logs into one parsing for both 2008 and 2010.
Added more error text messages based on id# from uploaded logs.

Known issues:
For 2010 roadster logs, the Sleep record does not contain the correct the min/max/brick voltages.
For 2010 roadster logs, the Charge30m record does not contain the motor temperature, so it is set to zero.
Brick numbers may be off by 1.

-Scott

hxxp://logparse.servebbs.com:10376 (change the xx to tt )
 
Last edited:
Paranoia will Destroy ya...

Scott,

Thanks for making your log analysis available to other owners.

The log files contain a bunch of information including your VIN, your driving second-by-second, your charging, and the GPS coordinates of everywhere you charge, which presumably includes your home location.

Although there's apparently no GPS data included with the driving records, I suspect that there is enough data that driving routes could be reconstructed by comparing driving patterns (speed, distance and stopping times) with stop lights and traffic flow data in your local area, thus potentially revealing a lot of personal information from those logs.

What's your data retention and privacy policy for the data that owners upload to your web server?


Paranoia will Destroy ya...:eek:

How many ways can someone get your address? Really?:rolleyes:
 
Standalone log parse tool available

I’ve decided to release my tesla log parsing tool. It is very basic. It will convert your tesla log file into an excel csv file. There are a lot of interesting values in the log like speed, current, torque, state of charge, etc. There are also a lot of values that are not decoded. I am releasing this tool to enlist your help to decode the remaining values. You can access the parser at the following website. http://logparse.servebbs.com:10376

Attached is the standalone version of my log_parse tool. I have included both a Windows and a Mac version. The web version will always have the most current version of the parser and should be used for unknown PERM_error messages (e.g. id3001).

Enjoy,

-Scott

Special thanks to Steve Casner who was very helpful in figuring out the scaling of many of the values used in the early developement of this tool. Tomsax figured out the brake/RGbrake bits used in the Drive1s record. And Thanks to all the other owners who provided log files and 2010 data logging sessions.
 

Attachments

  • Log_Parse_v13.zip
    108.2 KB · Views: 391
Last edited:
Will it ever go out as OpenSource?
Yes. At some point when I get tired of working on it... It's hack code though, because it was written to group bytes into words or long words (by identifying msB/lsB rollover). Later scaling was added to convert the data into meaningful values. Lastly a look up table was added to convert error message IDs into text messages. It may be more useful for me to publish the record types, values and scaling, so that someone could write a more user friendly (graphical) tool.

-Scott
 
Last edited:

As far as wireless TPMS, I don't think it's an issue because the TPMS sits on the instrument bus and the PEM, Motor, and battery sit on totally different busses. The only attack (DoS) you could send externally would be a flood of TPMS low pressure errors that might slow down the instrument bus and cause other errors. But you would still be able to pull over safely and stop the car.

-Scott
 
Hey Scott,

a) Thanks for the tool.
b) is a new version or source available?
This version expired on Thu Sep 23 11:29:29 2010, 1285262969

Sorry, I've been spending most of my spare time analyzing CAN bus data, I'll get a new version out in a week or two. I'm trying to release the source too, but I need to get the OK from work.

-Scott
 
Last edited:
Sorry, I've been spending most of time analyzing CAN bus data, I'll get a new version out in a week or two. I'm trying to release the source too, but I need to get the OK from work.

-Scott

Source would be great. I have a nice little data plotting utility that I wrote for another purpose, which would be fun to adapt to reading And displaying Roadster logs.