Category: Technology

Parsing JSON In JavaScript

We’ve been trying to get our BloomSky data parsed and reflected in OpenHAB — we can automatically turn the lights on when there is motion *and* the luminescence is lower than some desired value.  Bloomsky has an API which allows us to retrieve JSON formatted data from our weather station. I never worked with JSON before – I’d heard the term, but didn’t actually know what it was … but I needed to parse it in a JavaScript transform. Does JavaScript do JSON? D’oh! Turns out JSON is an abbreviation for JavaScript Object Notation, and JavaScript parses JSON data really well.

Still need to turn my example web code into a transform that runs from OpenHAB, but getting values out of a JSON formatted string is as easy as using the “parse” function:

<html>
	  <head>
	    <script>
	      function parseMyData() {
		var input = '{"DeviceID":"83237E","LAT":41.226644299999997,"LON":-81.7224322,"ALT":292.78720092773438,"UTC":-4,"DST":1,"Searchable":true,"RegisterTime":1464494138,"CityName":"Hinckley","StreetName":"Bellus Road","FullAddress":"Bellus Road, Hinckley, Ohio, US","DeviceName":"Buzzard Cam 01","BoundedPoint":null,"NumOfFollowers":5,"Data":{"Temperature":80.528000000000006,"ImageURL":"http://storage.googleapis.com/bloomsky-img/eaB1rJytnZSmm5y3qJ1krJqwmJmtoJU=.jpg","Humidity":50,"Night":false,"ImageTS":1465938980,"Luminance":3445,"TS":1465938980,"Rain":false,"Pressure":29.087148500000001,"Voltage":2613,"UVIndex":"1"},"Point":{},"VideoList":["http://storage.googleapis.com/bloomsky-video/eaB1rJytnZSmm5y3_-4_2016-06-09.mp4","http://storage.googleapis.com/bloomsky-video/eaB1rJytnZSmm5y3_-4_2016-06-10.mp4","http://storage.googleapis.com/bloomsky-video/eaB1rJytnZSmm5y3_-4_2016-06-11.mp4","http://storage.googleapis.com/bloomsky-video/eaB1rJytnZSmm5y3_-4_2016-06-12.mp4","http://storage.googleapis.com/bloomsky-video/eaB1rJytnZSmm5y3_-4_2016-06-13.mp4"],"NumOfFavorites":0}'

		var jsonOfInput = JSON.parse(input);

		document.write("<P>Device ID is: " + jsonOfInput.DeviceID + "</P>");
		document.write("<P>Temp is: " + jsonOfInput.Data.Temperature + "</P>");
		document.write("<P>Luminance is: " + jsonOfInput.Data.Luminance + "</P>");
	      }
	    </script>
	  </head>
	  <body>
	  <h2>Press the button to start</h2>
	    <input type="button" onclick="parseMyData()" value="Parse"/>
	  </body>
	</html>

Serving Custom Error Pages From Apache

At work, we are in the process of retiring an old password management web site. We want to direct users to the new site, and I don’t particularly want to handle each possible entry point an individual may have bookmarked. It seemed a lot quicker and easier to just move everything out of the directory and throw up a custom 404 page.

I am certain that I’ve used just “ErrorDocument ### /file.xtn” in Apache configurations to serve custom error pages, but when I set this up in our staging environment … I got the generic 404. Three days of Googling and reading Apache documentation later, and I have a configuration that actually serves a custom page when error 404 is encountered:

        ErrorDocument 404 /customized-404.html
        <Files "customized-404.html">
        <If "-z %{ENV:REDIRECT_STATUS}">
            RedirectMatch 404 ^/customized-404.html$

Voila, a pretty page that doesn’t in any way indicate 404 / not found / etc but rather says “hey, this web site is being retired. please go over yonder to manage your password.”.

Tracking Electrical Usage With SmartThings and AeonLabs Home Energy Meters

When we started shopping for solar generation installations, how much electricity we can consume was a challenging question. We were replacing our HVAC system, so “look at the last 12 months of electric bills” wasn’t an approach that would yield valid data. What we needed was a way to see electrical consumption for the next two or three weeks once our new HVAC system was installed.

We purchased several AeonLabs Home Energy Meters (HEM) and have been using them to track our power consumption for almost six months now. The HEM’s are set up in SmartThings & have a SmartApp-HEMLogger “SmartApp” attached to them that posts data to a MySQL table on our server via a web form (myURL needs to be … well, your URL).

Install a quick MySQL server (does not need to be Internet accessible) and a web server / programming language of your choice combination (*does* need to be Internet accessible – we are using Apache and PHP).

Create the database and a table to hold your energy data:

mysql> describe EnergyMonitors;
+-----------+-------------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-------------+------+-----+-------------------+----------------+
| keyID | int(11) | NO | PRI | NULL | auto_increment |
| monitorID | varchar(50) | YES | | NULL | |
| clampID | varchar(50) | YES | | NULL | |
| eventTime | timestamp | NO | | CURRENT_TIMESTAMP | |
| kwatts | double | YES | | NULL | |
| kwhours | double | YES | | NULL | |
+-----------+-------------+------+-----+-------------------+----------------+
6 rows in set (0.00 sec)

Create an ID within your database that has read/write permission to this table. I create another ID that has read-only access (pages displaying data use this ID, the page to post data uses the read/write ID).

We also track temperature — ideally, we’d be able to compare power consumption based on temperature *and* sunlight (we use a lot less power on a cold sunny day than I expect … just don’t know how much less) … but I’m not there yet. Currently, the weather database only holds temperature:

mysql> describe weather;
+--------------+-----------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-----------+------+-----+-------------------+-----------------------------+
| recordedTime | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| temperature | int(11) | YES | | NULL | |
| id | int(11) | NO | PRI | NULL | auto_increment |
+--------------+-----------+------+-----+-------------------+-----------------------------+
3 rows in set (0.01 sec)

Since we brought our Bloomsky online, I use the Bloomsky API to record temperature. Prior to that, I was parsing the XML from NWS’s closest reporting station’s – for Cleveland, that is  http://w1.weather.gov/xml/current_obs/KCLE.xml … there’s probably one near you. I grabbed both observation_time_rfc822 and temp_f in case observations were not updated regularly – the observation time was stored as the recordedTime. The temperature recording script is in cron as an hourly task.

You also need a small web page where SmartThings can post data — in PHP, I have

<?php
if(!$_POST["monitorID"] || !$_POST["clampID"] ){
echo "<html><body>\n";
echo "<form action=\"";
echo $_SERVER['PHP_SELF'];
echo "\" method=\"post\">\n";
echo "Monitor ID: <input type=\"text\" name=\"monitorID\"><br>\n";
echo "Clamp ID: <input type=\"text\" name=\"clampID\"><br>\n";
echo "KW Value: <input type=\"text\" name=\"kwatts\"><br>\n";
echo "KWH Value: <input type=\"text\" name=\"kWHours\"><br>\n";
echo "<input type=\"submit\">\n";
echo "</form>\n";
echo "</body></html>\n";
}
else{
$strMeter = $_POST["monitorID"];
$strClamp = $_POST["clampID"];
$strKWValue = $_POST["kwatts"];
$strKWHValue = $_POST["kWHours"];
print "<pre>Meter: $strMeter\nClamp: $strClamp\nKW: $strKWValue\nKWH: $strKWHValue\n</pre>\n";
$servername = "DatabaseHostname";
$username = "MySQLUID";
$password = 'MySQLPassword';
$dbname = "HomeAutomation";
$conn = new mysqli($servername, $username, $password, $dbname);
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
if($strKWHValue && $$strKWValue){
$sql = "INSERT INTO EnergyMonitors (monitorID, clampID, kwatts, kwhours) VALUES ('$strMeter', '$strClamp', '$strKWValue', '$strKWHValue')";
}
elseif($strKWHValue){
$sql = "INSERT INTO EnergyMonitors (monitorID, clampID, kwhours) VALUES ('$strMeter', '$strClamp', '$strKWHValue')";
}
else{
$sql = "INSERT INTO EnergyMonitors (monitorID, clampID, kwatts) VALUES ('$strMeter', '$strClamp', '$strKWValue')";
}
if ($conn->query($sql) === TRUE) {
echo "New record created successfully";
}
else {
echo "Error: " . $sql . "<br>" . $conn->error;
}

$conn->close();
}
?>

DatabaseHostname, MySQLUID and MySQLPassword need to be yours too. Since posted data is meant to come from a trusted source (a source where I’ve supplied the URL) and in a known good format, these are quick INSERT statements *not* the safest.

Check in your database that you are getting data, then let it run for a few hours. Once you have a little bit of history, you can start viewing your energy usage. Link it into Excel/Access using MyODBC for ad-hoc reporting, write your own code to do exactly what you want, use a generic charting package capable of reading MySQL data …

I am using pChart to display data – the chart I use most frequently is the stacked bar chart for kWH and a line chart for temperature. Here is the PHP code which generates this PNG: energyUsage-StackedBarChart-Flat – again, DatabaseHostname, MySQLUID, and MySQLPassword need to be yours.

StackedBarChart-KWHAndTemp

We no longer have our heat pump, air handler, and heat strips being monitored – but for periods where there is data from the other sources, we had several segments to our energy usage report (“other” is the report from the HEM on our mains MINUS all of the other reporting segments). You can yank all of the non-mains segments (or change them to be whatever sub-segments you are monitoring. The monitor ID comes from the HEM name in SmartThings – since those are user-configured, I have the names hard coded. You *could* hard code the Mains and then use “select distinct” to get a list of all the others and make the code more flexible.).

Short term charts (past 24 hours or so) can render out real-time, but longer term views take a long time to load. Since we’re looking more for trends and totals – being up to the second isn’t critical. I’ve got cron tasks that generate out PNG files of the charts.

Two enhancements for a nothing-doing rainy day – adding authentication to the HEM Logger (SmartThings posts from multiple netblocks, so there isn’t a good way to IP source restrict access to the post data page. Anyone with your URL could post rogue energy usage info into your database – or more likely try to hack your servers.) and add lumen to the weather data / graph displays.

Kerberos Authentication and LDAP Authorization In Apache

I’ve been authenticating users of Apache web sites against Active Directory using Kerberos for some time now. Installed krb5-workstation and mod_auth_kerb, configured /etc/krb5.conf for my specific domain, and added some config to the Directory section of the Apache config. Great if you just require valid-user (or require valid-user and then turn around and do some authorization within your web code using something like php_auth_user). Not so great, though, for restricting access to the site outside of web code. And I really didn’t want to code in an authorization function when my web server should be able to do that for me.

I FINALLY got kerberos authentication working in Apache with an LDAP authorization component. Turns out the  mod_auth_kerb version 5.1 that was available from the Yum repository is terribly buggy  – like not usable in this instance buggy. KrbLocalUserMapping did not consistently remove the realm component. I’d hit a site and it would know who I am, click a link and come across as me@REALM.TLD and get access denied errors, click refresh and get in because it knew I was me again. Or not. More than 50% failure rate.I built the 5.4 version from http://modauthkerb.sourceforge.net/ and haven’t had a problem since.

I’m authenticating to Active Directory using the Kerberos module then authorizing against a group housed in an external LDAP directory. You can totally point your LDAP config toward Active Directory & use AD groups instead:

AuthType Kerberos
AuthName “Kerberos AD Test”
KrbAuthoritative off
KrbMethodNegotiate on
KrbMethodK5Passwd on
KrbServiceName HTTP/this.isyour.url.tld@EXAMPLE.COM
KrbAuthRealms EXAMPLE.COM
KrbLocalUserMapping On
Krb5Keytab /path/to/keytabs/keytab.file

AuthBasicAuthoritative On
AuthBasicProvider ldap
AuthLDAPURL “ldaps://ldap.example.com/o=BaseDN?uid?sub?(&(cn=*))”
AuthLDAPBindDN “YOUR SERVICE ACCOUNT HERE”
AuthLDAPBindPassword “YOUR BIND PWD HERE”

AuthLDAPGroupAttribute uniqueMember
AuthLDAPGroupAttributeIsDN on
require ldap-group cn=Website Test,ou=groups,o=BaseDN

 

WooHoo! I hit the site from my domain-member computer, it knows I am LisaR. It then turns around and finds an LDAP user matching uid=LisaR and grabs the user’s fully qualified DN (because AuthLDAPGroupAttributesIsDN is ‘on’ here … if you are using just uids in your member list, that would be off). It then verifies that the fully qualified DN is a member of the Website Test group.

Now I’m trying to figure out how to let the user log in without supplying a realm (not everyone’s in the domain … and they need to be able to log in too. Works fine right now, provided they input their username as uid@REALM.TLD).

Same word, different meaning

I issue a lot of certificates from our internal company certificate authority – they’re free, and since I can publish trusted root signers out to the domain, they’re trusted to anyone who would be using the site. You can type pseudo-random values into your request and my CA will issue a certificate for you.
Today, though, I needed a certificate for a site that would be used by non-employees. People who are not subject to my domain GPO. People who do not trust my CA. So I did what everyone else does – got a real certificate 🙂
I generated my CSR (and actually typed in good data – my server is in Conway, AR, USA type location instead of Z or A). Went out to Verisign’s site … “The CSR contains an invalid state. Please click your browser’s Back button and enter a new CSR.”
Thought the CSR might have gotten corrupted somehow, so I tried again. Same result. Tried some different information – same result. Finally resorted to reading the instructions – locality names may not contain abbreviations. D’oh. State like organized political community not state like condition.

Missing The Point

A security researcher used a modified cat6 cable and default creds on airline seat electronic boxes to compromise flight control systems on an aircraft. That’s really bad, and the FBI is investigating the crime. But why is it that no one seems to care that (1) SEB’s ride on the same network as flight control systems, (2) there’s a default password no one has bothered to change, and (3) no one on the aircraft was in any way bothered by some dude digging around under the seat and messing with cables?

Seriously – in the system design meetings for a million dollar aircraft, someone thought it would be a good idea to save, what, a grand by having a single open network for all electronic components on the aircraft?!

And I sincerely hope the WiFi networks they’re starting to put on the aircraft are on an isolated network that has nothing to do with any of the flight control equipment. It’s one thing to notice a guy plugging into some box under his seat … a guy using his computer mid-flight, nothing to see there.

Linux Utility: xxd

When there’s something different between two files but you just cannot see it — xxd is a command line hex dump utility. You can even pair it with diff to see … oh, one has \r\n and the other just has \n

[lisa@FVP05 ljr]# diff -u <(xxd unix.txt) <(xxd dos.txt)
--- /dev/fd/63  2025-01-15 16:39:58.016083028 -0500
+++ /dev/fd/62  2025-01-15 16:39:58.018083031 -0500
@@ -1 +1 @@
-00000000: 4865 6c6c 6f21 0a                        Hello!.
+00000000: 4865 6c6c 6f21 0d0a                      Hello!..

Response Policy Zone (RPZ)

Years ago, Paul Vixie developed a component of the BIND DNS server that allowed server owners to easily override specific hostnames. We had done something similar for particularly bad hostnames — if your workstations use your DNS servers, you just have to declare yourself the name server for a domain that has the same name as the hostname you want to block (i.e. I become the NS record for forbidden.google.com and my clients are able to resolve all other records within the google.com zone, but when they resolve forbidden.google.com … they get whatever I provide). I usually did this to route traffic over a B2B VPN – provided the private IP address instead of the public IP provided by the domain owner’s name servers. But for a few really bad malware variants, I overrode their hostname. Problem was the technique wasn’t exactly easy. Every single host required a new DNS zone be created, configured on your DNS servers, and (at least in BIND) the service restarted.

Response Policy Zone was pushed as a functionality that would allow service providers (ISPs). That’s not a use case I forsee (it’s a lot of manual work), but it has become an important component of our company’s network security. Hosting an RPZ domain allows us to easily add new overrides for B2B VPN connected hosts. But it also means we can override hostnames that appear in phishing e-mail campaigns, malware hosts, infected web sites … basically anything we don’t want employees accessing.

Stopping clients from accessing infected sites is a great thing; but for hostnames that are indicative of a compromised box (i.e. there’s a difference between an employee clicking on a link within their e-mail that links them to a specific host and someone having malware on their box that automatically contacts a specific host), we set the IP address for the hostname to a honeypot.

The honeypot is bound to all unused ports on the host (there aren’t a lot of used ports on it), logs all contact to a database, then basically hangs the connection. We have a scheduled job that looks at the contact log and opens a ticket to the desktop support team to investigate the compromised host.

Baked French Toast Casserole with Maple Syrup

Ingredients

  • 1 loaf French bread (~16 ounces)
  • 8 large eggs
  • 1.5 cups heavy whipping cream
  • 1.5 cups milk
  • 2 tablespoons maple syrup
  • 1 teaspoon vanilla extract
  • 1/4 teaspoon ground cinnamon
  • 1/4 teaspoon ground nutmeg
  • Dash salt

Directions

Slice bread into 20 slices, 1-inch each. Arrange slices in a generously buttered 9 by 13-inch baking dish in 2 rows, overlapping the slices. In a large bowl, combine the eggs, half-and-half, milk, sugar, vanilla, cinnamon, nutmeg and salt and beat with a rotary beater or whisk until blended but not too bubbly. Pour mixture over the bread slices, making sure all are covered evenly with the milk-egg mixture. Spoon some of the mixture in between the slices. Cover with foil and refrigerate overnight.

The next day, preheat oven to 350 degrees F.

Bake for 40 minutes, until puffed and lightly golden. Serve with maple syrup, fresh fruit, and fresh whipped cream.

 

Anatomy of an LDAP Filter

LDAP filters are searches. Equality tests are supported for any attribute — attribute=value. Most attributes are case insensitive (but check your schema definition to verify!), and you can perform both exact matches (attribute=value) and sub-string matches (attribute=value*). While you will see * used in substring searches, LDAP filters do not support regex pattern matching. Some attributes support greater than and less than comparisons as well. A filter like (modifyTimestamp>=20140811000000Z) finds any object modified since 11 August 2014. This is useful when you are processing directory records and don’t want to look at any that haven’t changed since the last time your batch ran.

Tests are combined using Boolean operators. The three operators — AND (&), OR (|), and NOT (|) can be grouped and nested within parenthesis to from complex queries.

Your directory may support extensible matching — Active Directory does not. You may be able to find objects in the OUName OU using “ou:dn:=OUName”.

Your directory may support approximate matching — find close matches using “givenName=~Tim” where Tim and Timmy are returned.

To better understand an LDAP filter, decompose it into its sub-components. An example filter is:

(&(|(uid=e0*)(uid=n9*))(!(homeDirectory=*))(|(memberOf=cn=group1,ou=groups,o=example)(memberOf=cn=group2,ou=groups,o=example)))

This filter becomes

(&
     (|(uid=e0*)(uid=n9*))
     (!(homeDirectory=*))
     (|(memberOf=cn=group1,ou=groups,o=example)(memberOf=cn=group2,ou=groups,o=example))
)

The three sub-groups are AND’d — for an object to match the filter, it must meet all three sets of criterion.

Then decompose the first of the three sub-groups.

(|
     (uid=e0*)
     (uid=n9*)
)

This group uses an OR operator — the value of the uid attribute needs starts with e0 OR the value of the uid attribute starts with n9. You don’t need to use the same attribute in with the OR operator. I could use (|(st=OH)(title=Engineer)) to find all records where st is OH or title is Engineer.

The second sub-group has a single sub-component. The filter (homeDirectory=*) means “the value of homeDirectory is not NULL”. There is a NOT operator around the comparison — so we have the value of homeDirectory is NULL.

Decomposing the third sub-group:

(|
     (memberOf=cn=group1,ou=groups,o=example)
     (memberOf=cn=group2,ou=groups,o=example)
)

This group uses an OR operator — the value of memberOf is cn=group1,ou=groups,o=example OR the value of memberOf is cn=group2,ou=groups,o=example

The complete filter, then, finds records where

the value of the uid attribute needs starts with e0 OR the value of the uid attribute starts with n9

AND

the value of homeDirectory is NULL

AND

the value of memberOf is cn=group1,ou=groups,o=example OR the value of memberOf is cn=group2,ou=groups,o=example