Category: Technology

Ransomware

My company held a ransomware response through experiment recently – and, honestly, every ransomware response I’ve seen has been some iteration of “walk through backups until we find good files”. Maybe use something like the SharePoint versioning to help identify a good target date (although that date may be different for different files … who knows!). But why wouldn’t you attempt a proactive identification of compromised files?

The basis of ransomware is that it encrypts data and you get the password after paying so-and-so a bitcoin or three. Considering that NGO virus authors (e.g. those who aren’t trying to slow down Iran’s centrifuges) are generally interested in creating mayhem. There’s not a lot of disincentive to creating mayhem and making a couple of bucks. I don’t anticipate ransomware to become less prevalent in the future; in fact I anticipate seeing it in vigilante hacking: EntityX gets their files back after they publicly donate 100k to their antithesis organisation.

Since it’s probably not going away, it seems worthwhile to immediately identify the malicious data scrambling. Reverting to yesterday’s backups sucks, but not as much as finding that your daily backups have aged out and you’re stuck with the monthly backup from 01 Nov as your last “good” data set. It would also be good to merge whatever your last good backup is into the non-encrypted files so the only ‘stuff’ that reverts is a worthless scramble of data anyway. Sure someone may have worked on the file this morning and sucks for them to find their work back-rev’d to last night … but again that’s better than everyone having to reproduce their last two and a half months of work.

Promptly identifying the attack: There are routine processes that read changed files. Windows Search indexing, antivirus scanner, SharePoint indexing. Running against the Windows Search index log on every computer in the organisation is logistically challenging. Not impossible, but not ideal either. A central log for enterprise AV software or the SharePoint indexing log, however, can be parsed from the data centre. Scrape the log files for “unable to read this encrypted file” events. Then there are a myriad of actions that can be taken. Alert the file owner and have them confirm the file should be encrypted. Alert the IT staff when more than x encrypted files are identified in a unit time. Check the create time-stamp and alert the file owner for any files that were created prior to encountering them as encrypted.

Restoring only scrambled files: Since you have a list of encrypted files, you have a scope for the restore job. Instead of restoring everything in place (because who has 2x the storage space to restore to an alternate location?!). Restore just the recently identified as encrypted files – to an alternate location or in place. Ideally you’ve gotten user input on the encrypted files and can omit any the user indicated they encrypted too.

Scraping OpenHAB Karaf Console Data

Realized an easier way of scraping the Karaf console output – no need to SSH into the console (which, evidently, can timeout for inactivity … something I sort on my OpenSSH server with a config parameter whenever I’m looking to use tee and scrape output).

You can just pipe the startup script to tee. Have to push stderr into stdout to get the *errors* logged.

./start.sh 2>&1 | tee -a /tmp/logfile.txt

The output gets a little funky – maybe because of the color flags on some of the text? Dunno, but it’s grabbing the text and something like tail displays it without funky odd stuff

ESC[31m ESC[0m __ _____ ____ ESC[0m
ESC[31m ____ ____ ___ ____ ESC[0m/ / / / | / __ ) ESC[0m
ESC[31m / __ \/ __ \/ _ \/ __ \ESC[0m/ /_/ / /| | / __ | ESC[0m
ESC[31m/ /_/ / /_/ / __/ / / / ESC[0m__ / ___ |/ /_/ / ESC[0m
ESC[31m\____/ .___/\___/_/ /_/ESC[0m_/ /_/_/ |_/_____/ ESC[0m
ESC[31m /_/ ESC[0m 2.2.0-SNAPSHOTESC[0m
ESC[31m ESC[0m Build #1114 ESC[0m

Hit 'ESC[1m<tab>ESC[0m' for a list of available commands
and 'ESC[1m[cmd] --helpESC[0m' for help on a specific command.
Hit 'ESC[1m<ctrl-d>ESC[0m' or type 'ESC[1msystem:shutdownESC[0m' or 'ESC[1mlogoutESC[0m' to shutdown openHAB.

ESC[?1hESC=ESC[?2004hESC[36mopenhab>ESC[0m

But you get the java exceptions too:

      Exception in thread "pool-45-thread-5" java.lang.NullPointerException
              at java.util.AbstractCollection.addAll(AbstractCollection.java:343)
              at com.zsmartsystems.zigbee.ZigBeeNode.setNeighbors(ZigBeeNode.java:510)
              at com.zsmartsystems.zigbee.ZigBeeNetworkMeshMonitor$2.run(ZigBeeNetworkMeshMonitor.java:232)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)

 

Logging OpenHAB’s Karaf Console To A File

With OpenHAB2, there is a console where information is displayed. You can copy/paste from the console to save information, but if you are reproducing an issue and expect something to be logged, you can also dump the information from the console into a text file. This is done by ssh’ing into the Karaf console and using tee to write output to a file. Since the SSH server is bound to 127.0.0.1, you will need to use localhost or 127.0.0.1. This cannot be done remotely without some sort of firewall port redirection or OpenHAB change

     ssh UserName@localhost -p 8101 | tee -a /tmp/test.txt

So what’s the username? Karaf uses karaf as the username and password. OpenHAB uses the users.properties file (./openhab2/userdata/etc) to store users. Our file has the user openhab. You can google the default password or put your own crypt string in there and know the password.

Now everything that comes across the Karaf console (system output and stuff you type) will be in the /tmp/test.txt file.

[root@fedora01 ~]# tail -f /tmp/test.txt

                          __  _____    ____
  ____  ____  ___  ____  / / / /   |  / __ )
 / __ \/ __ \/ _ \/ __ \/ /_/ / /| | / __  |
/ /_/ / /_/ /  __/ / / / __  / ___ |/ /_/ /
\____/ .___/\___/_/ /_/_/ /_/_/  |_/_____/
    /_/                        2.2.0-SNAPSHOT
                               Build #1114

Hit '' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '' or type 'system:shutdown' or 'logout' to shutdown openHAB.

openhab> bundle:list
START LEVEL 100 , List Threshold: 50
 ID │ State    │ Lvl │ Version                │ Name
────┼──────────┼─────┼────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
──────────────────────────────────────────────────────────────────────────────────
 15 │ Active   │  80 │ 2.2.0.201712061711     │ ZWave Binding
 16 │ Active   │  80 │ 2.2.0.201712052342     │ ZigBee Binding
 17 │ Active   │  80 │ 5.3.1.201602281253     │ OSGi JAX-RS Connector
 18 │ Active   │  80 │ 2.4.5                  │ Jackson-annotations
 19 │ Active   │  80 │ 2.4.5                  │ Jackson-core
 20 │ Active   │  80 │ 2.4.5                  │ jackson-databind
 21 │ Active   │  80 │ 2.4.5                  │ Jackson-dataformat-XML
 22 │ Active   │  80 │ 2.4.5                  │ Jackson-dataformat-YAML
 23 │ Active   │  80 │ 2.4.5                  │ Jackson-module-JAXB-annotations
 24 │ Active   │  80 │ 2.7.0                  │ Gson
 25 │ Active   │  80 │ 18.0.0                 │ Guava: Google Core Libraries for Java
 26 │ Active   │  80 │ 3.0.0.v201312141243    │ Google Guice (No AOP)
 27 │ Active   │  80 │ 3.12.0.OH              │ nrjavaserial
 28 │ Active   │  80 │ 1.5.8                  │ swagger-annotations
 29 │ Active   │  80 │ 3.19.0.GA              │ Javassist
 31 │ Active   │  80 │ 3.5.2                  │ JmDNS
 34 │ Active   │  80 │ 1.1.0.Final            │ Bean Validation API
 36 │ Active   │  80 │ 2.0.1                  │ javax.ws.rs-api

Net Neutrality And Infrastructure Investments

Ajit Pai claims eliminating net neutrality will spur carriers to invest in network infrastructure. And he’s not exactly wrong – there’s equipment required to QOS traffic to allow companies who have paid access extortion to have their traffic move faster. There’s equipment required to block services for subscribers who haven’t opted to pay for, say, the “Social Media Bundle”. Billing systems will need to be updated, which means more work for developers.

Turning all of the public roadways over to private corporations and allowing them to elect to operate them as free or toll roadways would spur a lot of investment or hiring too. There’s not an automated toll collecting barricade at the end of my street today, or a human toll collector. Imprisoning half a percent of the entire US population spurred a lot of investment and hiring too – new prisons, guards, support staff.

Investment or hiring is not, eo ipso, a boon. Sure it’s great for the company whose products are being purchased. Sure it’s great for the person who just got a job. But for society some impetus for investment and hiring is outright detrimental.

Since Pai has outright stated that he cares naught for public opinion, I am appealing to my members of Congress to enact legislation to enact principals similar to the existing net neutrality regulations. That’s the point of checks and balances in government – the courts could deem the reclassification of Internet providers to be unconstitutional (it isn’t, so not gonna happen). Congress can pass laws changing that which the executive branch needs to enforce. The executive branch can veto the legislative net neutrality bill, but a 2/3 majority in Congress can override the veto. Courts can rule those laws unconstitutional (since the existing regulations have already passed legal challenges, that’s doubtful too).

Nothing Is New

I keep seeing articles hyping the anonymity of bitcoin-type “currency”. That’s not a new concept in value stores. Non-registered bearer bonds allowed untraceable fund transfers. As bearer instruments are not illegal in the United States, such bonds can still be issued. The holder cannot get any tax exemptions on interest paid for the bond, but you can transact business using bearer bonds. And just like bitcoin-type currencies … you’re screwed if someone takes it. Bonds provide legal recourse – bitcoin, not so much. If no one wants to pay a couple hundred thousand dollars for your bitcoin, you have little bits on disk. It’s like an anonymous stock — it’s worth whatever people are currently willing to pay for it.

As a data storage technique – distributed across the world, redundant, but ultimately meaningless in its sub-components to anyone who happens to have a snippet – it’s intriguing. But as a non-dodgy way of transacting business, it’s just silly.

The Colloquial Occam’s Razor

Occam’s razor – it is futile to do with more things that which can be done with fewer – is colloquially rendered as “the simplest solution is the most likely”. We had multiple tickets opened today for authentication failures on an Apache web server. Each malfunctioning site uses LDAP authentication and authorization against an Oracle Unified Directory. Nothing in the error logs. The service account from the Apache configuration can log in and query the directory from the box using ldapsearch, so the account is valid and there is nothing in the OUD preventing access from this particular host.

That’s a puzzler, and I was about to take down a lot of web sites to reload the service with its log level set to debug. Not even sure what made me do it, but I went out to the groups and looked at their member lists. Oops. Something had gone wrong with the identity management platform and employee accounts had been cleared from the groups (all of the contractors were still members, which made it even stranger). Added a few people back into groups appropriate for their position, voila they could log into their site again.

No idea how the identity management group restored the memberships, but verifying people who should have been members (who had been members and had done nothing to remove their memberships) were actually members of the group saved a lot of time running through debug logs. Sometimes the simplest answer is the most likely.

Apple FaceID

The irony of facial recognition — the idea is that you trade some degree of privacy for enhanced security. There are 10k four digit codes – a 1:10000 chance of any specific code unlocking your device. Apple touted a one in a million chance of facial recognition unlocking your phone.

So you trade your privacy for this one in a million super secure lock. Aaaaand a Vietnamese security firm can hack the phone with a mask. Not even a *good* mask (like I take a couple of your pictures, available online, synthesize them into a 3d image and print a realistic mask).

This feat wasn’t accomplished with millions of dollars of hardware. It took them a week and 150$ (plus equipment, but a 3d printer isn’t as expensive as you’d think).

Boyd v. United States or Riley v. California provide fourth amendment protection for phone content … but that only means the police need a warrant. Fourth amendment, check. Fifth amendment … Commonwealth of Virginia v. Baust  or  United States v. Kirschner says that you while cannot be compelled to reveal a passcode to allow police to access your phone (testimonial) … a fingerprint is not testimonial, it is documentary. And can be compelled. As with a lot of security, one can ask why I care. If I’m not doing anything wrong then who cares if the police peruse my phone. But if I’m not protesting, why do I care if peaceful assembly is being restricted. I’m not publishing the Paradise Papers, so why do I care if freedom of the press is being restricted? Like Martin Niemöller and the Nazis – by the time they get around to harming you, there’s no one left to care.

OpenHAB Cloud Installation Prerequisites

We started setting up the OpenHAB cloud server locally, and the instructions we had found omitted a few important steps. They say ‘install redis’ and ‘install mongodb’ without providing any sort of post-install configuration.

Redis
# This is optional – if you don’t set a password, you’ll just get a warning on launch that a password was supplied but none is required. While the service is, by default, bound to localhost … I still put a password on everything just to be safe

vi /etc/redis.conf # Your path may vary, this is Fedora. I've seen /etc/redis/redis.conf too

# Find the requirepass line and make one with your password

480 # requirepass foobared
requirepass Y0|_|RP@s5w0rdG03s|-|3re

# Restart redis

service redis restart

Mongo:
# Install mongo (dnf install mongo mongo-server)
# start mongodb

service mongod start

# launch mongo client

mongo

# Create user in admin database

db.createUser({user: "yourDBUser", pwd: "yourDBUserPassword", roles: [{role: userAdminAnyDatabase", db: "admin"}]});
exit

# Modify mongodb server config to use security

vi /etc/mongod.conf

# remove remarkes before ‘security: ‘ and ‘authorization’ – set authorization to enabled:

99 # secutiry Options - Authorization and other security settings
100 security:
101 # Private key for cluster authentication
102 #keyFile: <string>
103
104 # Run with/without security (enabled|disabled, disabled by default)
105 authorization: enabled

# restart mongo

service mongod restart

#Launch mongo client supplying username and connecting to the admin database

mongo -uyourDBUser -p admin

# it will connect and prompt for password – you can use db.getUser to verify the account (but you just logged into it, so that’s a bit redundant)

MongoDB shell version: 3.2.12
Enter password:
connecting to: admin
> > db.getUser("yourDBUser");
{
        "_id" : "admin.yourDBUser",
        "user" : "yourDBUser",
        "db" : "admin",
        "roles" : [
                {
                        "role" : "userAdminAnyDatabase",
                        "db" : "admin"
                }
        ]
}

# Create the openhab database — mongo is a bit odd in that “use dbname” will switch context to that database if it exists *and* create the databse if it doesn’t exist. Bad for typo-prone types!

use yourDBName;

# Create the user in the openhab database

db.createUser({user: "yourDBUser", pwd: "yourDBUserPassword", roles: [{role: readWrite", db: "yourDBName"}]});

# You can use get user to verify it works

db.getUser("yourDBUser");
exit

# Now you can launch the mongo client connecting to the openhab database:

mongo -uyourDBUser -p yourDBName

# It will prompt for password and connect. At this point, you can use “node app.js” to launch the openhab cloud connector. Provided yourDBUser, yourDBUserPassword, and yourDBName match what you’ve used in the config file … it’ll connect and create a bunch of stuff

 

Strange spam

We have been getting spam messages with the subject “top level quality of paint bucket” both at home and at work. I get that it costs essentially nothing to send a million junk e-mail messages, so it doesn’t take a lot of sales for a campaign to be profitable. But are there seriously people who buy their paint buckets from cold e-mails? Especially e-mails that I thought were trying to sell me buckets of paint.

And how lazy is a spam campaign that uses static strings in the subject field?

Containerized Development v/s Microservices

While both monolithic and microservice applications can be deployed in containers, there is a significant difference. Understanding that difference can save time/money/effort decomposing an application into microservices when the benefits you desire can be gained through simple containerized deployments.

One of the touted benefits of microservices — the ability for different teams to use different internal practices, different coding standards, hell even different languages and still have a functioning application because the interface is static and well documented … well, that sounds like a nightmare to me.

A company with which I worked a decade ago had teams of developers devoted to different components of the application — essentially your team owned a class or set of functions. The class/functions were had well documented and static interfaces — you wouldn’t change void functionX(int iVariable, string strOtherVariable) to return boolean values. Or to randomly add inputs (although functions were overloaded). Developers were tasked with ensuring backwards compatibility of their classes and functions. The company had a “shared libraries” development team who worked on, well, shared libraries. Database I/O stuff, authentication frameworks, GUI interfaces. A new project would immediately pull in the relevant shared functions, then start developing their code.

Developers were able to focus on a small component of the application, were able to implement code changes without having to coordinate with other teams, and consumers of their resource were able to rely on the consistent input and output of the functions as well as consistent representation of class objects.

When a specific project encountered resource shortfalls (be that family emergencies reducing workers or sales teams making overly optimistic commitments), the dozens of C# programmers could be shifted around to expand a team. In a team with an outstanding team lead, employees could easily move to other groups to progress their career.

What happens in a microservices environment? You’ve got a C# team, a Java team, a Python team. You get some guy in from Uni and he’s starting up a LISP team because Lisplets will get his code delivered through Tomcat. The next guy who comes in starts the F90 team because why not? Now I’m not saying someone with a decade of experience in Java couldn’t learn LISP … but you go back to “Google up how to do X in LISP” programming speed. There are language nuances of which you are not aware and you introduce inefficiency and possibly bugs to the code.

What’s my point? Well, (1) business practices (we program in this language, here’s our style guide, etc) are going to negate some of the perceived benefits of microservices. The small gain to be had by individual teams picking their own way are going to be outweighed by siloing (some guy from the Java team isn’t going to move into a lead role over on the C# team) and resource limitations (I cannot reallocate resources temporarily). But (2) you can architect your project to provide, basically, the same benefits.

Microservices make sense where an application has different components with different utilization rates. A product that runs a Super Bowl commercial may see a huge spike in web traffic — but scaling up thousands of complete web servers to handle the load is an inefficient use of resources. There’s a lot of product browsing, but shipping quotes, new account creations, and check-outs are not all scaling linearly to web hits. Adding tens of thousands of browsing components and only expanding the new-account-creation or checkout services as visitors decide to make purchases can be done more quickly to respond in real-time to traffic increases.

Applications where each component gets about the same amount of use … I use Kubernetes to manage a cluster of sendmail servers. As mail traffic increases, additional PODs are brought online. It’s a configuration I’d like to mirror at work — we currently have nine sendmail servers — to provide physical and site redundancy for both employee mail traffic and automated system traffic. With Kubernetes, three servers in each of the two sites (six total) would provide ample resources to accommodate mail flow. Automated systems send a lot of mail at night, and the number of pods servicing that VIP would increase. User mail flow increases during the day, so while automated mailflow pods would be spun down … user mail flow ones would be spun up. With a 33% reduction in servers, I’ve created a solution with more capacity for highly used functions (this function could be the primary usage of all six servers) that is geo-redundant (one of the current systems is *not* geo-redundant as the additional two servers in the alternate site couldn’t be justified). But I didn’t need to decompose sendmail into microservices to achieve this. Simply needed to build a containerized sendmail.