Tag Archives: raspberry pi

STRATUX ADS-B Receiver – open ports inventory

While working on a service to read data published by the STRATUX open source ADS-B receiver, it became a big of a guessing game regarding which ports where open and what protocol they might support.

In the event others are interested, there is the the port inventory from a STRATUX v1.6r1 device built on 03-MAR-2021

PortProtocolServiceAnalysis
80tcphttp Golang net/http server AngularJS web interfae
8080tcphttp-proxy says Dump1090 but always returns 404 error
9977tcphttpJson Pi System Info – temp, memory, etc.
30001tcppago-services ?-TBD-
30002tcppago-services ?appears to be a string of hashstrings (e.g.: *8DAB1D28990C37BB78042B8E5676; )
30003tcpbasestationannounced as “ADS-B flight data”
30004tcplistenerThis is a binary “Beast” input port
30005tcpunknown-TBD- possibly some binary data stream (e.g.: ‘JXf#<' )
30006tcpADS JsonJson of partial ADS-B data – always seems to be missing loation and speed info
example:
{
  "Icao_addr": 11403422,
  "DF": 17,
  "CA": 5,
  "TypeCode": 4,
  "SubtypeCode": 6,
  "SBS_MsgType": 1,
  "SignalLevel": 0.000344,
  "Tail": "LEXUS05 ",
  "Squawk": null,
  "Emitter_category": 6,
  "OnGround": false,
  "Lat": null,
  "Lng": null,
  "Position_valid": false,
  "NACp": null,
  "Alt": null,
  "AltIsGNSS": false,
  "GnssDiffFromBaroAlt": null,
  "Vvel": null,
  "Speed_valid": false,
  "Speed": null,
  "Track": null,
  "Timestamp": "2021-03-05T17:16:47.203Z"
}
30104tcpunknown -TBD- possibly raw dump1090 data stream

Diving into the Port Data


Port 30002

Text stream of data, available via a tcp connection. Each message is a hash of some sort, value of which is undetermined at this point.

Here is an example of the message stream

*5DA8C3EA379B4F;
*02E195B70936B4;
*5DA21603EB2A06;
*02C60BB129C2D0;
*02C60BB129C2D0;
*8DAD21C1592DD7EF9896504E8E50;
*02E19718EEFD09;
*8DA1B52B596233C53D9901456D9D;
*8DA216035913378FE2327C198F8A;
*8DA2160399104987B8340BFCE929;
*8DA9942E585F100DBA20C5712D44;
*8DA9942E9908212C509814A6DF7A;

Port 30003

Text stream of data, available via a tcp connection. Each message is comma delimited, and appears to the the ICAO integer converted to Base16 hex.

Here is an example of the message stream

MSG,3,111,11111,A4AB64,111111,2021/03/06,17:16:35.554,2021/03/06,17:16:35.573,,8900,,,30.41666,-98.63536,,,,,,0 MSG,4,111,11111,A4AB64,111111,2021/03/06,17:16:35.554,2021/03/06,17:16:35.574,,,159,93,,,-448,,,,,0 MSG,3,111,11111,A313BF,111111,2021/03/06,17:16:35.587,2021/03/06,17:16:35.626,,36000,,,29.68039,-98.32875,,,,,,0 MSG,7,111,11111,A8DB58,111111,2021/03/06,17:16:35.602,2021/03/06,17:16:35.627,,5900,,,,,,,,,,0 MSG,3,111,11111,A2A18A,111111,2021/03/06,17:16:35.607,2021/03/06,17:16:35.627,,5625,,,29.66141,-98.47789,,,,,,0 MSG,4,111,11111,A2A18A,111111,2021/03/06,17:16:35.607,2021/03/06,17:16:35.628,,,221,248,,,128,,,,,0 MSG,7,111,11111,A8DB58,111111,2021/03/06,17:16:35.608,2021/03/06,17:16:35.628,,5900,,,,,,,,,,0 MSG,3,111,11111,A417D6,111111,2021/03/06,17:16:35.609,2021/03/06,17:16:35.628,,24000,,,30.09300,-97.62418,,,,,,0

Port 30005

Binary stream of data, available via a tcp connection.

Here is an example of the message stream

y# ??2??)k?????3]?Sx?l3??.?@?????B?xf~3??.??yRX??L?ma??3??/?8 q?!I????3??1Q???? Z?O??3??69???+(X??????-N2??6??”ᕸ?eP3??7??#??o???,???3??;W#??o#?t?l??Zg3??>?&???Y}?o? ???I3??C k??+(???P,2N?3??D????yR? ??-3?3??F,???Y??x?_??2??F6? ?8\`3??Hg???+?ȱ?/?2??I????p2??Ke`#ᕸ?eP2??P?]?+(??A2??U^ ?8\`2??Un<ᕸ???3??XWW???.X?????_?3??X????Y???{?+3??X?h%??o?B?`_?_b?2??Z?f???~?2??Z?j??p2??]- ??|+?3??_e? ??SxX??56S?J?3??_?????.@ ???2??`????2??e?F%]?o7C?3??jt??Sx?y%?`2??l/???? 3??n??????? ??-??3??q?U q? ?-0?Jc3??q?)???Y??)? T??2????7?8?}?2???γ 8إ?2????H ?FU3????|????Y??1??2???L?]?\g??2????E]??.???3??|?{??????>?0 2????]?\g??3????????!;

Port 30006

High volume text stream, available via tcp connection. Messages are in a Json format. Observation is that these always seem to lack geo-location, speed and altitude data.

Here is an example of the message stream

{“Icao_addr”:10627880,”DF”:11,”CA”:5,”TypeCode”:0,”SubtypeCode”:0,”SBS_MsgType”:8,”SignalLevel”:0.000816,”Tail”:null,”Squawk”:null,”Emitter_category”:null,”OnGround”:false,”Lat”:null,”Lng”:null,”Position_valid”:false,”NACp”:null,”Alt”:null,”AltIsGNSS”:false,”GnssDiffFromBaroAlt”:null,”Vvel”:null,”Speed_valid”:false,”Speed”:null,”Track”:null,”Timestamp”:”2021-03-06T17:34:12.129Z”} {“Icao_addr”:10645249,”DF”:11,”CA”:5,”TypeCode”:0,”SubtypeCode”:0,”SBS_MsgType”:8,”SignalLevel”:0.000846,”Tail”:null,”Squawk”:null,”Emitter_category”:null,”OnGround”:false,”Lat”:null,”Lng”:null,”Position_valid”:false,”NACp”:null,”Alt”:null,”AltIsGNSS”:false,”GnssDiffFromBaroAlt”:null,”Vvel”:null,”Speed_valid”:false,”Speed”:null,”Track”:null,”Timestamp”:”2021-03-06T17:34:12.139Z”} {“Icao_addr”:11402414,”DF”:4,”CA”:0,”TypeCode”:0,”SubtypeCode”:0,”SBS_MsgType”:5,”SignalLevel”:0.001636,”Tail”:null,”Squawk”:null,”Emitter_category”:null,”OnGround”:false,”Lat”:null,”Lng”:null,”Position_valid”:false,”NACp”:null,”Alt”:6075,”AltIsGNSS”:false,”GnssDiffFromBaroAlt”:null,”Vvel”:null,”Speed_valid”:false,”Speed”:null,”Track”:null,”Timestamp”:”2021-03-06T17:34:12.142Z”} {“Icao_addr”:10645249,”DF”:11,”CA”:5,”TypeCode”:28,”SubtypeCode”:1,”SBS_MsgType”:8,”SignalLevel”:0.000837,”Tail”:null,”Squawk”:null,”Emitter_category”:null,”OnGround”:false,”Lat”:null,”Lng”:null,”Position_valid”:false,”NACp”:null,”Alt”:null,”AltIsGNSS”:false,”GnssDiffFromBaroAlt”:null,”Vvel”:null,”Speed_valid”:false,”Speed”:null,”Track”:null,”Timestamp”:”2021-03-06T17:34:12.158Z”}

Stratux – repartition to use entire sd card

The issue..

It’s been an ongoing tail of woe, when it comes to installing a new version of STRATUX on a new card; out of the box, at least in all the installs I’ve done, it fails to make use of the entire card. With any sort of logging enabled to measure performance, run tests or just keep even a short history of contacts received.. you’re out of disc space… FAST:

Here is the ‘df’ dump of a STRATUX fixed position station running the latest version (released September 2019). As you can see, it’s only allocated 2GB of the 32GB card’s total available space. That is a lot of wasted space on the card.

Filesystem      Size  Used Avail Use% Mounted on
/dev/root       1.8G  1.6G  110M  94% /
/dev/mmcblk0p1   60M   20M   41M  34% /boot

Get total device size:

fdisk -l | grep Disk

Disk /dev/ram0: 4 MiB, 4194304 bytes, 8192 sectors
...
Disk /dev/mmcblk0: 29.7 GiB, 31914983424 bytes, 62333952 sectors

Disklabel type: dos
Disk identifier: 0xe6a544c8

Repartitioning the Device

With the physical partition located.. start fdisk:
fdisk -u /dev/mmcblk0

I like to increase the size of the main partition to 6G to leave room for installing more system updates and tools.

To do this you will need to know the starting and ending blocks of the partition. That is available with the ‘print’ command:

Command (m for help): p

Results:

Disk /dev/mmcblk0: 29.7 GiB, 31914983424 bytes, 62333952 sectors
Units: sectors of 1 * 512 = 512 bytes
Disk identifier: 0xe6a544c8

Device         Boot  Start     End Sectors  Size Id Type
/dev/mmcblk0p1        8192  131071  122880   60M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      131072 3887103 3756032  1.8G 83 Linux

To increase the size, the partition must first be deleted, then re-create at the exact same starting block, or the filesystem will become corrupted.

First, delete the partition with the ‘d’ command, selecting partition #2.

Next, re-create the partition with the same starting block, but now with increased filesystem size:

Command (m for help): d 

  Partition number (1,2, default 2): 2

  Partition 2 has been deleted.

Command (m for help): p
 
  Device         Boot Start    End Sectors Size Id Type
  /dev/mmcblk0p1       8192 131071  122880  60M  c W95 FAT32 (LBA)


Command (m for help): n

  Partition type
     p   primary (1 primary, 0 extended, 3 free)
     e   extended (container for logical partitions)
  Select (default p): p
  Partition number (2-4, default 2): 2

First sector (2048-62333951, default 2048): 131072

Last sector, +sectors or +size{K,M,G,T,P} (131072-62333951, default 62333951): +6G

  Created a new partition 2 of type 'Linux' and of size 6 GiB.

Command (m for help): w
  
  The partition table has been altered.
  Calling ioctl() to re-read partition table.
  Re-reading the partition table failed.: Device or resource busy

  The kernel still uses the old table. The new table will be used
  at the next reboot or after you run partprobe(8) or kpartx(8).

The partition table will have been modified but the kernel will not be able to take that into account as some partitions are mounted.

In theory, the command ‘partx /dev/mmcblk0’ is all that is required.. however I’ve found that rebooting is the only way to really reload the partition, so that the filespace can be increased.

reboot

Once system comes back up, run ‘resize2fs‘ to expand the filesystem.

Fill up drive with current filesystem

Execute `resize2fs` and run an on-line expansion of the filesystem, and finally verify it again with ‘df -h’

resize2fs /dev/mmcblk0p2

  resize2fs 1.43.3 (04-Sep-2016)
  Filesystem at /dev/mmcblk0p2 is mounted on /; on-line resizing
  required
  old_desc_blocks = 1, new_desc_blocks = 1
  The filesystem on /dev/mmcblk0p2 is now 1572864 (4k) blocks long.

Running df shows that it has in fact resized. STEP 1 COMPLETED

root@STRATUX-FIXED:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       5.9G  1.6G  4.2G  28% /
devtmpfs        459M     0  459M   0% /dev
tmpfs           463M     0  463M   0% /dev/shm
tmpfs           463M  6.2M  457M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           463M     0  463M   0% /sys/fs/cgroup
/dev/mmcblk0p1   60M   20M   41M  34% /boot

Creating A Personalized Recreation Dashboard – Setting up the Pi – Part 1

With the impending move to one of the state’s premier recreation areas, I’m embarking upon setting up a customized data systems based on the Raspberry Pi, that will collect and display recreation related data. Some of the goals of this project are:

THE PLAN

  • River flow rate — knowing what the current river flow helps decided if it will be a weekend of tubing, kayaking or finding something else to do. River flow is managed by the Army Corps of Engineers and can vary greatly. Knowing the current flow helps plan the day.
  • Lake Level — certain boat ramps, beaches and parks are better at near-full level than not. Having a good idea of what the beaches and marinas might be like before heading down the hill to the lake can be helpful.
  • Area Traffic — this one will be tricky, but the plan is to leverage TXDOT traffic APIs to find out what the traffic might be like heading into the major cities (ex. Austin or San Antonio) should there be a need to do so. There are a lot of existing resources but having it all in a dashboard would be nice to have.
  • WX — of course.. some sort of simple weather forecast dash to show the day’s expected temps, if lightning is in the forecast (you don’t want to be on the river/lake when the heavens open up), etc.

THE RECREATION DEVICE

The plan is to use a Raspberry Pi 3b as the data harvesting and webapp platform. In addition to the standard Pi, I’ve added a 3.5″ touchscreen to display the data in a stand-alone format.

Raspberry Pi 3b with 3.5″ touchscreen

The full plan for the webapp will support both the micro 3.5″ display as well as interface versions for larger displays such as a table or desktop.

CONFIGURING THE PI

Once you have downloaded the Pi Image you want to run, have it burned to the micro SD card and the system booted up, you’ll need to have it either plugged into your network on an ethernet cable, or attached via HDMI to a monitor.

I plugged the stripped down device into the router with short cable:

This image has an empty alt attribute; its file name is stratux-pi-initial.jpg

Once it was powered on, checked the router to locate it’s DHCP IP address, in this case it’s 192.168.1.220

Using the terminal window on my, I ssh’d into the device as the user ‘pi’ using default password ‘raspberry’ . Once connected, I set about updating the PI.

sudo apt-get upgrade

If you face a dpkg error.. this command will resolve it, and you can re-run the command above.

sudo dpkg --configure -a

sudo apt-get upgrade

SETTING UP WIFI

To untether the device from my LAN, you’ll need to configure the Pi to connect to your WiFi. If you are running the desktop version of Raspberian, you can do this with the networking setup dialogs. If you do not, here are the steps for setting up your networking, including defining your network and password.

Locate the wifi network configuration file and add the following data. You can of course use tools to encrypt the password in this file, but I’m not super-concerned about an exploit on my LAN so I’ve let this in plain text.

sudo vi /etc/wpa_supplicant/wpa_supplicant.conf

Add the following block, with your network specifics

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

country=USnetwork={
  ssid="YOUR-NETWORK-SSID"
  psk="your-super-secret-password-goes-here"
  key_mgmt=WPA-PSK
}

Now setup the wlan with a static IP address. You could configure a DNS service to automagically detect the IP address of the system, but for me it was easier for me to pick a static IP and configure the Pi’s network accordingly.

Edit the network/interfaces file:

sudo vi /etc/network/interfaces

Add the following block. This will set a static IP (make sure it’s in a range not managed by your DHCP to avoid future conflicts).

auto wlan0

iface wlan0 inet static
  metric 0
  wireless-power off
  hostname recserver
  wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
  address 192.168.1.100
  netmask 255.255.255.0
  gateway 192.168.1.1
  network 192.167.1.0
  broadcast 192.168.1.255
  dns-nameservers 192.168.1.1 8.8.8.8

SETTING UP THE DISPLAY

The display that I have installed in the Pi’s case is the MHS-3.5 display. Installing the drivers requires only a few commands. While logged in as the user ‘pi’ download the LCD-show package from git and configure execute the display configuration script.

git clone https://github.com/Lcdwiki/LCD-show.git

Move to the cloned directory and run the setup script

cd LCD-show
sudo ./MHS35-show

This will download and install packages required to use the display. Once it completes, you’ll want to reboot the Pi to verify that it is working properly.

sudo init 6

Upon reboot, the screen should become visible and operable.

In Part 2, I’ll cover installing Node, Maven and Node Package Manager.

Installing CasperJS on Raspberry Pi

This is generally a simple process to install CapserJS on a *NIX box, but had not done this on a Raspberry Pi before, and ran into a couple of minor gotchas. This is a stripped down explanation of what I needed to do, but it will server as a reference for me, and maybe for some other people as well.

Prerequisites for installing CapserJS

  • Node
  • npm
  • PhantomJS

Node

Installing node is very simple, but you should grab root access to do it. This should be simple and run without any issues.

apt-get install nodejs


npm

The Node Package Manager will be used later to actually install CasperJS, once PhantomJS has been successfully installed.

apt-get install npm


PhantomJS

PhantomJS is a web automation framework, that CasperJS uses to run it’s extended suite of tools. NOTE. I found that I needed to modify my user’s (pi in this case) environment to tell Phantom that not to expect a console for rendering webpages; I’ll document that at the bottom.

apt-get install phantomjs


CasperJS

Installing CasperJS is done most simply via the npm (Node Package Manager):

npm install -g casperjs


Handling the PhantomJS Display Error

pi@raspberrypi:~ $ casperjs
QXcbConnection: Could not connect to display
PhantomJS has crashed. Please read the bug reporting guide at
and file a bug report.
Aborted

The fix is to add this line to the .bashrc file, which will stop the error. I simply added this to the last line of the file.

export QT_QPA_PLATFORM=offscreen

STRATUX – Add swap and logging partitions.

It’s back! The STRAUX project lives again. With a fresh build of Stratux, it’s time to re-configure the SD card for swap space (new!) and a large logging partition to keep a full filesystem from crashing the device.

Get Current Partition Information

Once logged into the Straux box, switch to the root user and interrogate the disk partitions.

pi@raspberrypi: sudo su -
root@raspberrypi: fdisk -l
[...]

Device Boot Start End Sectors Size Type
/dev/mmcblk0p1 8192 131071 122880 60M W95 FAT32 (LBA)
/dev/mmcblk0p2 131072 3700592 3569521 17.G Linux

The important aspects you will want to record as the End block ids for each partition, this is important in the next step, creating more partitions.

Creating the Extended Partition

First task is I want to create the extended partition that can be used for swap. This will eventually contain the swap and log partitions.

Running the fdisk tool, start to edit the partition. You will want to reference the base block of the device.. eg: /dev/mmcblk0

root@raspberrypi: fdisk /dev/mmcblk0

Command (m for help): p

Device Boot Start End Sectors Size Type
/dev/mmcblk0p1 8192 131071 122880 60M W95 FAT32 (LBA)
/dev/mmcblk0p2 131072 3700592 3569521 17.G Linux

Command (m for help): n

Partition type
p primary (2 primary, 0 extended)
l logical (numbered from 5)
Select (default p): l

At this point, the tool threw an error adding partition 5…

Partition 5 is already defined. Delete it before re-adding it

Listing the partitions shows this:

Device Boot Start End Sectors Size Type
/dev/mmcblk0p1 8192 131071 122880 60M W95 FAT32 (LBA)
/dev/mmcblk0p2 131072 3700592 3569521 1.7G Linux
/dev/mmcblk0p3 3700593 62333951 5863d359 28G Extended
/dev/mmcblk0p5 2191 4194446 4192256 2G Linux

Re-running ‘n option and letting it setup partition 6, got me where I wanted to be. This process didn’t seem like it worked right.. but the end result matches up with my goal of a huge partition and a smaller 2G to be used for swap:

Command (m for help): n

Partition type
p primary (2 primary, 0 extended)
l logical (numbered from 5)
Select (default p): l

Adding logical partition 6
First Sector: 4198400
Last Sector: 62333951

Created a new partition 6 of type 'Linux and a size of 27.7 GiB

Command (m for help): p
Device Boot Start End Sectors Size Type
/dev/mmcblk0p1 8192 131071 122880 60M W95 FAT32 (LBA)
/dev/mmcblk0p2 131072 3700592 3569521 1.7G Linux
/dev/mmcblk0p3 3700593 62333951 5863d359 28G Extended
/dev/mmcblk0p5 2191 4194446 4192256 2G Linux
/dev/mmcblk0p6 4198400 62333951 58135552 27.7G Linux

Command (m for help): w

.. this is where the Ending block of your p2 partition number comes into play. The starting sector will be the ending number of your last partition (3569521) + 1 for next sector:


First Sector: 3700593
Last Sector: 62333951 (this was the default/max)

Created a new partition 3 of type 'Extended' and of size 28 GiB

Command (m for help): n

Partition type
p primary (2 primary, 0 extended)
l logical (numbered from 5)
Select (default p): l

Adding the largest partition, for log file storage

Now repeat this process to add the 2nd new partition, this one will consume the rest of the device, and eventually will be where the `/log` directory is mounted.

root@raspberrypi: fdisk /dev/mmcblk0

Command (m for help): p

Device Boot Start End Sectors Size Type
/dev/mmcblk0p1 8192 131071 122880 60M W95 FAT32 (LBA)
/dev/mmcblk0p2 131072 3700592 3569521 1.7G Linux
/dev/mmcblk0p3 3700593 7895039 4194447 2G Extended

Command (m for help): n

Partition type
p primary (2 primary, 0 extended)
l logical (numbered from 5)
Select (default p): l

First Sector: 4194448

AvMet Beta Release – STRATUX Database Reporter Project (open source!)

AvMet Beta Available on GitHub


For that last 6 months I have been capturing Aviation Mode-S and ADS-b data transmissions using my own customizd STRATUX Aviation Traffic Receiver running Raspberry Pi 3b. With my modifications, the device automatically shuts down daily for a few seconds to release the current SQLite3 database, and the start up pointing at a fresh database. Once released, the database is compressed to save space and for later download (I use rsync to automate that as well).

These database files can be very big, and contain hundreds of thousands of contact events each day. When these databases are generated every day, it doesn’t take long to get a backlog of them to examine for interesting things, such as NASA aircraft and some mysterious high-flyers using spoofed identities.

Being a professional programmer and data guru, this seemed like a perfect project to open-source and use a marketing tool for my consulting. After a week of design, development and preliminary tests.. it’s ready for comment.

What it does:


The tool is purpose built for my needs, so it might not (yet) be easily used on your own Stratux database files, but I feel I’ve made a reasonable effort in the interests of ease.

Each database file is opened and checked for a number of conditions and data:

  • Verify database contains expected STRATUX data tables.
  • Select first timestamp record and report it out. Since the name of the file might not be self descriptive, this is really helpful to know the dataset’s start time:
    Using Database File: './sqlite-stratux-temp'
    	Start: 2017-10-10 08:11:09.967 +0000 UTC
    	-----------------------------------------
    
  • Check dataset for obvious duplicates and remove them. It’s not uncommon for the message log to repeat a fixed position up to 5 times. Although this does not affect the current metrics, this bloats the dataset. There is an optional flag to disable this feature
  • Once duplicates are removed, each airframes dataset is checked for gross errors, such as changes in altitude or speed that would be possible only with alien spacecraft; records are also removed. There is an optional flag to disable this feature
      10486213 -- Bad Distance  	   24924 --> 51857   
      10617769 -- Bad Distance  	   82039 --> 206496  
      10707621 -- Bad Speed              318 --> 8303
    
  • Following data repair a few metrics are pulled from the dataset, looking for interesting boundary events such as Fastest and Furthest contacts. Each record reports the Callsign or Tail Registration number (when available), and the Mode-S ICAO24 code as well as altitude, speed and distance for the specific event. Example:
    	  FASTEST:   XAOLE [0D0AAE]   43000 ft.  @   529 kts.      23 mi.
    	  SLOWEST:   NDU45 [A6EBAF]    1150 ft.  @    61 kts.    2.26 mi.
    	  HIGHEST:   XAOLE [0D0AAE]   47025 ft.  @   505 kts.      92 mi.
    	   LOWEST:  N41218 [A4DE2A]     550 ft.  @    77 kts.    1.71 mi.
    	  CLOSEST:  N6464R [A87E79]    2525 ft.  @    84 kts.    0.00 mi.
    	 FURTHEST:  N229NN [A203A9]   36000 ft.  @   472 kts.     128 mi.
    

    Here is an example of an interesting contact captured last year along the coast of California; a high-altitude signal from a NASA 747 test aircraft almost 100 miles off the coast.

    	  HIGHEST: NASA747 [AA0DB8]   43025 ft.  @   473 kts.      32 mi.
    	 FURTHEST: NASA747 [AA0DB8]   43000 ft.  @   466 kts.      92 mi.
    
  • During this phase of the process, special Squawk Code events are trapped that might indicate special civil, science or military operations. Example:
    	**ALERT:  N7253N [A9B8ED] 4403     2500 ft.  @   118 kts.  SR-71, YF-12, U-2 and B-57, pressure suit flights
    	**ALERT:  N7272N [A9C038] 4402     2075 ft.  @   116 kts.  SR-71, YF-12, U-2 and B-57, pressure suit flights
    	**ALERT:  N7274N [A9C07E] 4404     1250 ft.  @   100 kts.  SR-71, YF-12, U-2 and B-57, pressure suit flights
    	**ALERT:  N907CH [AC8967] 4442     1800 ft.  @   140 kts.  SR-71, YF-12, U-2 and B-57, pressure suit flights
    

    In the above case, N7253N, N7272N and N7274N are Department of Homeland Security Border Patrol helicopter contacts.

Here is a recent report example:

        Start: 2017-10-10 08:11:09.967 +0000 UTC
        -----------------------------------------
          FASTEST: AFR6721 [3951C1]   32975 ft.  @   531 kts.      14 mi.
          SLOWEST:  N42894 [A51D07]     950 ft.  @    51 kts.    3.21 mi.
          HIGHEST:    JCB1 [43E9D5]   47025 ft.  @   469 kts.      34 mi.
           LOWEST:  N724DP [A9B2C1]     400 ft.  @    72 kts.    1.65 mi.
          CLOSEST:  N605CH [A7DA0C]   16200 ft.  @   400 kts.    0.02 mi.
         FURTHEST:  N9023N [AC7968]   33050 ft.  @   433 kts.     143 mi.

Where can you get it?

You can fork or pull the current source code from GitHub here: IngeniiCode AvMet

Stratux Webserver – what’s behind the scenes?

Stratux has a rich community of forums, and a lot of information about debugging Strtux, but so far good hacking information is really hard to find. One of the things of most interest to me was “What is severing up this webpage?”

Every search was a dead end, so I went back to my *NIX system administration roots and thought.. “Well, if someone wont admit what’s serving up the stream.. I’ll find out for myself.

Who’s Your Server? — gen_gdl90

A couple of quick commands told me which PID was hanging onto Port 80 and from there which process was associated with the PID:


root@raspberrypi:~# fuser 80/tcp
80/tcp: 3110
root@raspberrypi:~# ps aux | grep 3110
root 3110 51.9 2.9 973132 27600 ? Ssl 15:19 47:03 /usr/bin/gen_gdl90
root 6606 0.0 0.2 4276 1904 pts/0 S+ 16:50 0:00 grep 3110
What files does it have open?

Once I had an idea of who’m I was looking for, running lsof with the port number gave me 60 entries… and bingo.. there was the nugget of gold I was looking for:


root@raspberrypi:~# lsof -p 3110
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
gen_gdl90 3110 root cwd DIR 179,2 4096 2 /
[...]
gen_gdl90 3110 root 35u IPv6 1849628 0t0 TCP StratuxWiFi.io:http->10.100.0.188:49157 (ESTABLISHED)
gen_gdl90 3110 root 36u IPv6 210216 0t0 TCP StratuxWiFi.io:http->10.100.0.188:64155 (ESTABLISHED)
gen_gdl90 3110 root 37u IPv6 1859641 0t0 TCP StratuxWiFi.io:http->10.100.0.188:49165 (ESTABLISHED)
gen_gdl90 3110 root 38u IPv6 1759876 0t0 TCP StratuxWiFi.io:http->10.100.0.188:65420 (ESTABLISHED)
gen_gdl90 3110 root 41u IPv6 1778784 0t0 TCP StratuxWiFi.io:http->10.100.0.188:65442 (CLOSE_WAIT)
gen_gdl90 3110 root 48r REG 179,2 146998 51146 /var/www/maui/js/angular.min.js
The Web Path

Once onto the trail of the web path, I see that this is an Angular based application (ugh.. I really despise Angular.. I just do.), and all based on some JS stuffs. I get it. for an app like this the two-way data binding of Angular is probably the right too; but I still do not (no do I have to) like it.


root@raspberrypi:~# cd /var/www/maui/
root@raspberrypi:/var/www/maui# ls -l
total 12
drwxr-xr-x 2 root root 4096 Mar 15 2016 css
drwxr-xr-x 2 root root 4096 Mar 15 2016 fonts
drwxr-xr-x 2 root root 4096 Mar 15 2016 js
root@raspberrypi:/var/www/maui#

What I was hoping to find was the location of the that status page.. but.. I believe that what I’m looking for now is the .JS file that manages that label. Initially this looked like a dead end…


root@raspberrypi:/var/www/maui# egrep -r Distance *
root@raspberrypi:/var/www/maui#

Realiazing this was some templating sub-directory, and the root was likely at /var/www, I ran another search that found the location of the desired string, and likely the location of the parts I’m looking for:


root@raspberrypi:~# cd ..
root@raspberrypi:/var/www# egrep -lnr 'Distance' *

plates/js/traffic.js
plates/traffic-help.html
plates/traffic.html

STRATUX – Filesystem Full; Managing disk space redux

Checking in with my Starux project this morning, I found it unresponsive. A physical check shows a flashing red light on the Pi… something has gone haywire, and I couldn’t SSH into the little thing, so a really hard cold restart was in order.

Following the restart I quickly shut Stratux back down to start performing diagnostics:


pi@raspberrypi:~ $ sudo su -

root@raspberrypi:~# service stratux stop

Diagnostics 101

Check Filesystem Health

The last time the little Stratux suddenly had problems it was a filesystem space issue ( previous article ). It turns out the boot drive space was OK but the new Logging partition was again, completely consumed:

Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/root        1815440 1391284    331164  81% /
devtmpfs          469688       0    469688   0% /dev
tmpfs             474004       0    474004   0% /dev/shm
tmpfs             474004    6340    467664   2% /run
tmpfs               5120       4      5116   1% /run/lock
tmpfs             474004       0    474004   0% /sys/fs/cgroup
/dev/mmcblk0p1     61384   20400     40984  34% /boot
/dev/mmcblk0p4   8125880 8109496         0 100% /var/log

Locate the Culprit

There are a number of ways to locate large file on a *NIX system. My favorite tool is find. First thing I want to do is locate any file that is larger than 1 Gigabyte, and sure enough it located a massive sqlite database file. The same one that ate up all the space on the boot drive. So.. this is going to require some more extrodinary measures to maintain 100% 24×7 operational status.


root@raspberrypi:~# cd /var/log
root@raspberrypi:/var/log# find . -size +1G -exec ls -l {} \;
-rw-r--r-- 1 root root 7463211008 Apr 29 21:48 ./stratux.sqlite
Start Solving

First order of business is to move aside the massive database, but try to preserve the data for examination. Since the filesystem is full, I can’t zip this thing in place, so first some space needs to cleared on the device. The things I’m least interested in go first.. like the zipped syslogs and any other ‘archived’ file (those with a .# suffix).

-rw-r----- 1 root adm      728756 Apr 25 06:25 syslog.5.gz
-rw-r----- 1 root adm     2552845 Apr 26 06:25 syslog.4.gz
-rw-r----- 1 root adm     2447263 Apr 27 06:25 syslog.3.gz
-rw-r----- 1 root adm     2498089 Apr 28 06:25 syslog.2.gz
-rw-r----- 1 root adm    33735478 Apr 29 06:25 syslog.1
-rw-r----- 1 root adm       54307 Apr 30 05:19 debug.1
-rw-r----- 1 root adm      380928 Apr 30 05:19 kern.log.1
-rw-r----- 1 root adm   332365824 Apr 30 06:18 daemon.log.1
-rw-r----- 1 root adm       48723 Apr 30 06:25 auth.log.1
-rw-r----- 1 root adm           0 Apr 30 06:25 syslog.1.gz
-rw-r----- 1 root adm     1032192 Apr 30 06:25 messages.1

root@raspberrypi:/var/log# rm -f *.gz *\.[0-9]

But.. that’s not going to doe the complete trick, especially if after deleting files df still shows 100% utilization. You need to figure out what is holding which deleted file(s).

Normallyh, the best way to do that on *NIX is with lsof. Much to my chagrin, it was not available on the OS… so I had to go grab it. Thankfully I had the main filesystem on a different partition that still had enough space to install more tools! Once lsof was installed, grep through the list of open file handles and find those marked for delete.. and those are the processes that have handles pointing to those files. However, this didn’t help me with Rasperian Jessie. I know that a reboot will recover the space so.. that was the next step.. a brutal warm-boot.


root@raspberrypi:/var/log# lsof
-su: lsof: command not found

root@raspberrypi:/# apt-get install lsof
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
libperl4-corelibs-perl
The following NEW packages will be installed:
libperl4-corelibs-perl lsof
...

root@raspberrypi:/var/log# lsof | grep deleted
root@raspberrypi:/var/log#

root@raspberrypi:/var/log# init 6

Archiving Massive SQLite Database

The first thing was to move aside the current database, and then restart stratux to verify it can created a new empty database for it’s purposes.. then shut it right back down again.


root@raspberrypi:/var/log# mv stratux.sqlite stratux.sqlite.1
root@raspberrypi:/var/log# service stratux start
root@raspberrypi:/var/log# ls -l
total 7291480
[...]
-rw-r--r-- 1 root root 4096 Apr 30 14:27 stratux.sqlite
-rw-r--r-- 1 root root 7466160128 Apr 30 14:24 stratux.sqlite.1
[...]
root@raspberrypi:/var/log# service stratux stop

A new empty database file has been created [ 4096 Apr 30 14:27 stratux.sqlite ]. This tells me that moving aside the current database file on a periodic basis, compressing and then archiving it should be sufficient to maintain operational status.

Logging Insanity

Before restarting Stratux, I zerod out these log files. Running for first a few moments these files were already reading up a lot of space. Tailing one of them I see that Stratux, with my current settings is logging A LOT of data to these log files. I feel this was my first error.. enabling too much logging. My settings look like this:

With those settings enabled, there are a lot of GPS and other events that I don’t really have a use for, being dumped into the Statux log.


-rw-r--r-- 1 root root 1361778 Apr 30 14:39 stratux.log

Turning OFF ‘Verbose Message Log’ made that insanity stop.

Replay logging is what is writing to the SQLite database. So the question is. how much of that data do I want to keep, and how much will I lose if I turn off the replay logs. I think that will be research for another day… right now the goal is to recover disk space by compressing the massive database file that was moved aside, and get Stratux stabilized again. Once compressed, check filesystem and file size!


root@raspberrypi:/var/log# gzip stratux.sqlite.1

root@raspberrypi:/var/log# df

Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/root        1815440 1392384    330064  81% /
devtmpfs          469688       0    469688   0% /dev
tmpfs             474004       0    474004   0% /dev/shm
tmpfs             474004   12272    461732   3% /run
tmpfs               5120       4      5116   1% /run/lock
tmpfs             474004       0    474004   0% /sys/fs/cgroup
/dev/mmcblk0p4   8125880  681792   7008276   9% /var/log
/dev/mmcblk0p1     61384   20400     40984  34% /boot

root@raspberrypi:/var/log# ls -lktr --color --block-size=M stratux.sqlite*

-rw-r--r-- 1 root root 643M Apr 30 14:24 stratux.sqlite.1.gz
-rw-r--r-- 1 root root   8M Apr 30 15:21 stratux.sqlite
-rw-r--r-- 1 root root   1M Apr 30 15:22 stratux.sqlite-shm
-rw-r--r-- 1 root root   5M Apr 30 15:22 stratux.sqlite-wal

With the file compressed, it could be copied elsewhere for analysis.