I spent entirely too much time today setting up my development environment on my Windows laptop. All the tools installed fine, but there were a lot of post-install configuration problems with Git Bash and ssh. Eventually I got everything – Python, pip, pipenv/virtualenv, git, and VSCode – installed and working, but it was a frustrating experience.
I’m very new to Python, and it turns out I’m more lacking in the fundamentals than I thought. I started working on the data ingestion code for my rtlamr data visualization project (which I’ll refer to as rtlamrvis from now on). I’m so new to Python, in fact, that I forgot how some simple things work, such as reading from a file, and had to look up some code examples. I coded for over an hour, but it felt like a good chunk of that time was spent looking at documentation.
Previous to Python, all my recent coding was in Perl. Not knowing enough of the basics of Python means that I keep trying to work with and think about data structures the way I would in Perl. The data I’m trying to read from rtlamr is in JSON format, and each line contains a nested dictionary. Here are a few sample lines:
{"Time":"2019-01-30T21:02:00.39401341-08:00","Offset":0,"Length":0,"Message":{"ID":12345678,"Type":12,"TamperPhy":3,"TamperEnc":0,"Consumption":364968,"ChecksumVal":63704}}
{"Time":"2019-01-30T21:18:35.642387991-08:00","Offset":0,"Length":0,"Message":{"FrameSync":5795,"ProtocolID":30,"EndpointType":156,"EndpointID":23456789,"Consumption":128898,"Tamper":2568,"PacketCRC":959}}
{"Time":"2019-01-30T21:28:21.728291686-08:00","Offset":0,"Length":0,"Message":{"FrameSync":5795,"ProtocolID":30,"EndpointType":156,"EndpointID":34567890,"Consumption":209850,"Tamper":3080,"PacketCRC":59910}}
{"Time":"2019-01-30T21:32:22.551431986-08:00","Offset":0,"Length":0,"Message":{"FrameSync":5795,"ProtocolID":30,"EndpointType":156,"EndpointID":45678901,"Consumption":128902,"Tamper":2568,"PacketCRC":57215}}
{"Time":"2019-01-30T21:38:51.098216075-08:00","Offset":0,"Length":0,"Message":{"ID":56789012,"Type":12,"TamperPhy":1,"TamperEnc":0,"Consumption":339342,"ChecksumVal":21221}}
Looking at my commit from earlier today doesn’t reveal what I had been doing wrong. I had changed so much while trying to figure out how to access the nested elements of the dictionary, I ended up removing all my original code that was throwing syntax errors. After a break from staring at the code, and some dinner, I made another attempt at it, and this time got it to work. I think one of my problems previously was that I was trying to use curly braces instead of square brackets:
import json
rtlamr_file = "testdata/test.json"
rtlamr_data = []
with open(file=rtlamr_file) as rtlamr_f:
for line in rtlamr_f:
line_json = json.loads(line)
# this works
if 'ID' in line_json['Message']:
print('Found ID')
elif 'EndpointID' in line_json['Message']:
print('Found EndpointID')
# this does NOT work
if 'ID' in line_json{'Message'}:
print('Found ID')
elif 'EndpointID' in line_json{'Message'}:
print('Found EndpointID')
The moral of the story: Python is not Perl. Also, it’s late and I must sleep…hopefully this blog post will still make some sense in the morning.