Category: Technology

Setting up redis sandbox

To set up my redis sandbox in Docker, I created two folders — conf and data. The conf will house the SSL stuff and configuration file. The data directory is used to store the redis data.

I first needed to generate a SSL certificate. The public and private keys of the pair are stored in a pem and key file. The public key of the CA that signed the cert is stored in a “ca” folder.

Then I created a redis configuation file — note that the paths are relative to the Docker container

################################## MODULES #####################################

################################## NETWORK #####################################
# My web server is on a different host, so I needed to bind to the public 
#   network interface. I think we'd *want* to bind to localhost in our
#   use case. 
# bind 127.0.0.1
# Similarly, I think we'd want 'yes' here
protected-mode no

# Might want to use 0 to disable listening on the unsecure port
port 6379


tcp-backlog 511
timeout 10
tcp-keepalive 300
################################# TLS/SSL #####################################
tls-port 6380

tls-cert-file /opt/redis/ssl/memcache.pem
tls-key-file /opt/redis/ssl/memcache.key
tls-ca-cert-dir /opt/redis/ssl/ca

# I am not auth'ing clients for simplicity
tls-auth-clients no
tls-auth-clients optional

tls-protocols "TLSv1.2 TLSv1.3"
tls-prefer-server-ciphers yes
tls-session-caching no

# These would only be set if we were setting up replication / clustering
# tls-replication yes
# tls-cluster yes
################################# GENERAL #####################################
# This is for docker, we may want to use something like systemd here. 
daemonize no
supervised no

#loglevel debug
loglevel notice

logfile "/var/log/redis.log"
syslog-enabled yes
syslog-ident redis
syslog-facility local0

# 1 might be sufficient -- we *could* partition different apps into different databases
# But I'm thinking, if our keys are basically "user:target:service" ... then report_user:RADD:Oracle
# from any web tool would be the same cred. In which case, one database suffices. 
databases 3
################################ SNAPSHOTTING  ################################
save 900 1
save 300 10
save 60 10000

stop-writes-on-bgsave-error yes

rdbcompression yes
rdbchecksum yes

dbfilename dump.rdb


# 
dir ./

################################## SECURITY ###################################
# I wasn't setting up any sort of authentication and just using the facts that
#  (1) you are on localhost and
#  (2) you have the key to decrypt the stuff we stash
#  to mean you are authorized. 

############################## MEMORY MANAGEMENT ################################
# This is what to evict from the dataset when memory is maxed
maxmemory-policy volatile-lfu
############################# LAZY FREEING ####################################

lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
lazyfree-lazy-user-del no

############################ KERNEL OOM CONTROL ##############################
oom-score-adj no
############################## APPEND ONLY MODE ###############################

appendonly no
appendfsync everysec

no-appendfsync-on-rewrite no

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

aof-load-truncated yes

aof-use-rdb-preamble yes

############################### ADVANCED CONFIG ###############################
hash-max-ziplist-entries 512
hash-max-ziplist-value 64

list-max-ziplist-size -2

list-compress-depth 0

set-max-intset-entries 512

zset-max-ziplist-entries 128
zset-max-ziplist-value 64

hll-sparse-max-bytes 3000

stream-node-max-bytes 4096
stream-node-max-entries 100

activerehashing yes

client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60

dynamic-hz yes

aof-rewrite-incremental-fsync yes

rdb-save-incremental-fsync yes

########################### ACTIVE DEFRAGMENTATION #######################
# Enabled active defragmentation
activedefrag no

# Minimum amount of fragmentation waste to start active defrag
active-defrag-ignore-bytes 100mb

# Minimum percentage of fragmentation to start active defrag
active-defrag-threshold-lower 10

Once I had the configuration data set up, I created the container. I’m using port 6380 for the SSL connection. For the sandbox, I also exposed the clear text port. I mapped volumes for both the redis data, the SSL files, and the redis.conf file

docker run --name redis-srv -p 6380:6380 -p 6379:6379 -v /d/docker/redis/conf/ssl:/opt/redis/ssl -v /d/docker/redis/data:/data -v /d/docker/redis/conf/redis.conf:/usr/local/etc/redis/redis.conf -d redis redis-server /usr/local/etc/redis/redis.conf --appendonly yes

Voila, I have a redis server ready. Quick PHP code to ensure it’s functional:

<?php

$sodiumKey   = random_bytes(SODIUM_CRYPTO_SECRETBOX_KEYBYTES); // 256 bit
$sodiumNonce = random_bytes(SODIUM_CRYPTO_SECRETBOX_NONCEBYTES); // 24 bytes

#print "Key:\n";
#print sodium_bin2hex($sodiumKey);
#print"\n\nNonce:\n";
#print sodium_bin2hex($sodiumNonce);
#print "\n\n";

$redis = new Redis();
$redis->connect('tls://memcached.example.com', 6380); // enable TLS
//check whether server is running or not
echo "<PRE>Server is running: ".$redis->ping()."\n</pre>";

$checks = array(
    "credValueGoesHere",
        "cred2",
        "cred3",
        "cred4",
        "cred5"
);

#$ciphertext = safeEncrypt($message, $key);
#$plaintext = safeDecrypt($ciphertext, $key);

foreach ($checks as $i => $value) {
    usleep(100);
    $key = 'credtest' . $i;
    $strCryptedValue =  base64_encode(sodium_crypto_secretbox($value, $sodiumNonce, $sodiumKey));
    $redis->setEx($key, 1800, $strCryptedValue);        // 30 minute timeout
}

echo "<UL>\n";
for($i = 0; $i < count($checks); $i++){
        $key = 'credtest'.$i;
        $strValue = sodium_crypto_secretbox_open(base64_decode($redis->get($key)),$sodiumNonce, $sodiumKey);
        echo "<LI>The value on key $key is: $strValue \n";
}
echo "</UL>\n";

echo "<P>\n";
echo "<P>\n";
echo "<UL>\n";
$objAllKeys = $redis->keys('*');        // all keys will match this.
foreach($objAllKeys as $objKey){
        print "<LI>The key $objKey has a TTL of " . $redis->ttl($objKey) . "\n";
}
echo "</UL>\n";
foreach ($checks as $i => $value) {
    usleep(100);
        $value = $value . "-updated";
    $key = 'credtest' . $i;
    $strCryptedValue =  base64_encode(sodium_crypto_secretbox($value, $sodiumNonce, $sodiumKey));
    $redis->setEx($key, 60, $strCryptedValue);          // 1 minute timeout
}

echo "<UL>\n";
for($i = 0; $i < count($checks); $i++){
        $key = 'credtest'.$i;
        $strValue = sodium_crypto_secretbox_open(base64_decode($redis->get($key)),$sodiumNonce, $sodiumKey);
        echo "<LI>The value on key $key is: $strValue \n";
}
echo "</UL>\n";


echo "<P>\n";
echo "<UL>\n";
$objAllKeys = $redis->keys('*');        // all keys will match this.
foreach($objAllKeys as $objKey){
        print "<LI>The key $objKey has a TTL of " . $redis->ttl($objKey) . "\n";
}
echo "</UL>\n";



foreach ($checks as $i => $value) {
    usleep(100);
        $value = $value . "-updated";
    $key = 'credtest' . $i;
    $strCryptedValue =  base64_encode(sodium_crypto_secretbox($value, $sodiumNonce, $sodiumKey));
    $redis->setEx($key, 1, $strCryptedValue);          // 1 second timeout
}


echo "<P>\n";
echo "<UL>\n";
$objAllKeys = $redis->keys('*');        // all keys will match this.
foreach($objAllKeys as $objKey){
        print "<LI>The key $objKey has a TTL of " . $redis->ttl($objKey) . "\n";
}
echo "</UL>\n";

sleep(5); // Sleep so data ages out of redis
echo "<UL>\n";
for($i = 0; $i < count($checks); $i++){
        $key = 'credtest'.$i;
        $strValue = sodium_crypto_secretbox_open(base64_decode($redis->get($key)),$sodiumNonce, $sodiumKey);
        echo "<LI>The value on key $key is: $strValue \n";
}
echo "</UL>\n";


?>

Memcached with TLS in Docker

To bring up a Docker container running memcached with SSL enabled, create a local folder to hold the SSL key. Mount a volume to your local config folder, then point the ssl-chain_cert and ssl_key to the in-container paths to the PEM and KEY file:

 

docker run -v /d/docker/memcached/config:/opt/memcached.config -p 11211:11211 –name memcached-svr -d memcached memcached -m 64 –enable-ssl -o ssl_chain_cert=/opt/memcached.config/localcerts/memcache.pem,ssl_key=/opt/memcached.config/localcerts/memcache.key -v

Using Memcached in PHP

Quick PHP code used as a proof of concept for storing credentials in memcached — cred is encrypted using libsodium before being send to memcached, and it is decrypted after being retrieved. This is done both to prevent in-memory data from being meaningful and because the PHP memcached extension doesn’t seem to support SSL communication.

<?php

# To generate key and nonce, use sodium_bin2hex to stash these two values
#$sodiumKey   = random_bytes(SODIUM_CRYPTO_SECRETBOX_KEYBYTES); // 256 bit
#$sodiumNonce = random_bytes(SODIUM_CRYPTO_SECRETBOX_NONCEBYTES); // 24 bytes

# Stashed key and nonce strings
$strSodiumKey = 'cdce35b57cdb25032e68eb14a33c8252507ae6ab1627c1c7fcc420894697bf3e';
$strSodiumNonce = '652e16224e38da20ea818a92feb9b927d756ade085d75dab';
# Turn key and nonce back into binary data
$sodiumKey = sodium_hex2bin($strSodiumKey);
$sodiumNonce = sodium_hex2bin($strSodiumNonce);

# Initiate memcached object and add sandbox server
$memcacheD = new Memcached;
$memcacheD->addServer('127.0.0.1','11211',1);  # add high priority weight server added to memcacheD

$arrayDataToStore = array(
    "credValueGoesHere",
        "cred2",
        "cred3",
        "cred4",
        "cred5"
);

# Encrypt and stash data
for($i = 0; $i < count($arrayDataToStore); $i++){
    usleep(100);
    $strValue = $arrayDataToStore[$i];
    $strMemcachedKey = 'credtest' . $i;
        $strCryptedValue =  base64_encode(sodium_crypto_secretbox($strValue, $sodiumNonce, $sodiumKey));
    $memcacheD->set($strMemcachedKey, $strCryptedValue,time()+120);
}

# Get each key and decrypt it
for($i = 0; $i < count($arrayDataToStore); $i++){
        $strMemcachedKey = 'credtest'.$i;
        $strValue = sodium_crypto_secretbox_open(base64_decode($memcacheD->get($strMemcachedKey)),$sodiumNonce, $sodiumKey);
        echo "The value on key $strMemcachedKey is: $strValue \n";
}
?>

Updating git on Windows

There’s a convenient command to update the Windows git binary

git update-git-for-windows

Which is useful since ADO likes to complain about old git clients —

remote: Storing packfile... done (51 ms)
remote: Storing index... done (46 ms)
remote: We noticed you're using an older version of Git. For the best experience, upgrade to a newer version.

Python Selenium Headed v/s Headless

We are automating a file download — it works fine when running headed, but headless execution doesn’t manage to log in. Proxying the requests through Fiddler show that several JavaScript pages download unexpected content.

I’ve added a user-agent to the request, but I’ve noticed that the ChromeDriver also sets sec-ch-* headers … I expect the null sec-ch-ua causes the web server to refuse our request. I don’t see any issues in the ChromeDriver repo for the sec-ch-* headers … and I don’t really want to walk back versions until I find one that doesn’t try setting this header value. Firefox’s GeckoDriver, though, doesn’t set them … so I moved the script over to use Firefox instead of Chrome and am able to download the file.

Headed run:

GET /o/telx-theme/css/A.bootstrap.css+slick,,_slick.css,Mcc.JKqfH-juDS.css.pagespeed.cf.ZO22sEGAvO.css HTTP/1.1
Host: example.com
Connection: keep-alive
sec-ch-ua: “Chromium”;v=”92″, ” Not A;Brand”;v=”99″, “Google Chrome”;v=”92″
sec-ch-ua-mobile: ?0
User-Agent: “Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36
Accept: text/css,*/*;q=0.1
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: no-cors
Sec-Fetch-Dest: style
Referer: https://example.com/web/guest/login
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: JSESSIONID=0330C2C988F31010790779A126EA6F55.node1; COOKIE_SUPPORT=true; GUEST_LANGUAGE_ID=en_US; AWSELB=039B496118DDEAD697B2B51C93852940763289C324F9E7C7223F953330AF5506573D13C4D5599541FD3CADB645303C1CAEB6D26992826965DA6C8BEDBDE9C297AE26CD76ED; AWSELBCORS=039B496118DDEAD697B2B51C93852940763289C324F9E7C7223F953330AF5506573D13C4D5599541FD3CADB645303C1CAEB6D26992826965DA6C8BEDBDE9C297AE26CD76ED; TS0194d418=01092b79076749232d762d2a6c232e015d103453fbeda3826bd3d20e1d937f5a90cabe03655c97a79198969eea539e4c2e7fc426216092c78ccda85763d52300ce05672704e45b4fc25516d2c24279656db7b0242f7c8b9c8bfed35b7608afb0c54bbc33d489f431059d048094c1e707a20d28031885ca6c61f81613ac299044f0c2b9ba36

 

Headless run:

GET /o/telx-theme/css/A.bootstrap.css+slick,,_slick.css,Mcc.JKqfH-juDS.css.pagespeed.cf.ZO22sEGAvO.css HTTP/1.1
Host: example.com
Connection: keep-alive
sec-ch-ua:
sec-ch-ua-mobile: ?0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36
Accept: text/css,*/*;q=0.1
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: no-cors
Sec-Fetch-Dest: style
Referer: https://example.com/web/guest/login
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US
Cookie: JSESSIONID=F4293ECE33B134CC368C0E62D6923B48.node1; COOKIE_SUPPORT=true; GUEST_LANGUAGE_ID=en_US; AWSELB=039B496118DDEAD697B2B51C93852940763289C324A5AB24AE470C70960B5319A93C181302D27B4C9425A4AA05795334C4404D491FBCC8E6A9B809746A802EAC2EC8C2FBFA; AWSELBCORS=039B496118DDEAD697B2B51C93852940763289C324A5AB24AE470C70960B5319A93C181302D27B4C9425A4AA05795334C4404D491FBCC8E6A9B809746A802EAC2EC8C2FBFA; TS0194d418=01ba3b12a4ef612e3839114024b5082fd19d56b17293c914ff867740ad37ae362e385934695ad3fc275074bfd1ee24c7d1591b146ad39d153a8758aecc8eb44d374dc1c689e540deca9566f723df65e9f5ad26551e25bacd5df14e4e6104a91a0ecdb59a65176bd5a0ebed284847e0e6618a05ed1d9db6b544e195d8e1f41164e7199a6596

Fortify on Demand Remediation: Command Injection

Any time user input is used to shell out and execute a command, you risk the user executing more than you want. I can string together commands in DOS using &, in Unix using ; … and stringing together commands and then executing them can blow things up spectacularly.

You can add any sort of filter to the user input to sort this … however, it doesn’t absolutely mean the vulnerability doesn’t exist. If your “user” input is trusted (in this case, it’s an automated process where some code calls some other code … so “passing” is good enough), no big. But if there are actual users involved, you should also filter out any characters that are used to string commands together.

Fortify on Demand Remediation — Cross-Site Scripting DOM (JS)

This vulnerability occurs when you accept user input or gather input from a AJAX call to another web site and then use that input in output. The solution is to sanitize the input, but Fortify on Demand seems to object strenuously to setting innerHTML … so filtering alone may not be sufficient depending on how you subsequently use the data.

To sanitize a string in JavaScript, use a function like this:

/**
 * Sanitize and encode all HTML in a string
 * @param  {string} str  The input string
 * @return {string} –    The sanitized string
 */
 var sanitizeHTML = function (str) {
    return str.replace(/&/g‘&amp;’).replace(/</g‘&lt;’).replace(/>/g‘&gt;’);
};

This will replace ampersands and the < and > from potential HTML tags with the HTML-encoded equivalents. To avoid using innerHTML, you might need to get a little creative. In many cases, I have a span where the results are displayed. I color-code the results based on success/failure … in that case, I an replace innerHTML with a combination of setting the css color style element to ‘green’ or ‘red’ then setting the innerText to my message string.

I can bold an entire element using a similar method. Changing some of the text, however … I haven’t come up with anything other than breaking the message into multiple HTML elements. E.g. a span for “msgStart”, one for “msgMiddle”, and one for “msgEnd” – I can then bold “msgMiddle” and set innerText for all three elements.