{"id":8210,"date":"2021-09-03T16:29:20","date_gmt":"2021-09-03T21:29:20","guid":{"rendered":"https:\/\/www.rushworth.us\/lisa\/?p=8210"},"modified":"2021-09-22T12:11:47","modified_gmt":"2021-09-22T17:11:47","slug":"cyberark-performance-improvement-proposal-in-memory-caching","status":"publish","type":"post","link":"https:\/\/www.rushworth.us\/lisa\/?p=8210","title":{"rendered":"Cyberark Performance Improvement Proposal \u2013 In-memory caching"},"content":{"rendered":"

Issue: <\/em><\/strong>The multi-step process of retrieving credentials from CyberArk introduce noticeable latency on web tools that utilize multiple passwords. This occurs each execution cycle (scheduled task or user access).<\/p>\n

Proposal: <\/em><\/strong>We will use a redis server to cache credentials retrieved from CyberArk. This will allow quick access of frequently used passwords and reduce latency when multiple users access a tool.<\/p>\n

Details: <\/em><\/strong><\/p>\n

A redis server will be installed on both the production and development web servers. The redis implementation will be bound to localhost, and communication with the server will be encrypted using the same SSL certificate used on the web server.<\/p>\n

Data stored in redis will be encrypted using libsodium. The key and nonce will be stored in a file on the application server.<\/p>\n

All password retrievals will follow this basic process:<\/p>\n

\"\"<\/a><\/p>\n

Outstanding questions:<\/p>\n

    \n
  1. Using a namespace for the username key increases storage requirement. We could, instead, use allocate individual \u2018databases\u2019 for specific services. I.E. use database 1 for all Oracle passwords, use database 2 for all FTP passwords, use database 3 for all web service passwords. This would reduce the length of the key string.<\/li>\n
  2. Data retention. How long should cached data live? There\u2019s a memory limit, and I elected to use a least frequently used algorithm to prune data if that limit is reached. That means a record that\u2019s fused once an hour ago may well age out before a frequently used cred that\u2019s been on the server for a few hours. There\u2019s also a FIFO pruning, but I think<\/em> we will have a handful of really frequently used credentials that we want to keep around as much as possible.Basically infinite retention with low memory allocation \u2013 we could significantly limit the amount of memory that can be used to store credentials and have a high (week? weeks?) expiry period on cached data.Or we could have the cache expire more quickly \u2013 a day? A few hours? The biggest drawback I see with a long expiry period is that we\u2019re retaining bad data for some time after a password is changed. I conceptualized a process where we\u2019d want to handle authentication failure by getting the password directly from CyberArk and update the redis cache \u2013 which minimizes the risk of keeping the cached data for a long time.<\/li>\n
  3. How do we want to encrypt\/decrypt stashed data? I used libsodium because it\u2019s something I used before (and it\u2019s simple) \u2013 does anyone have a particular fav method?<\/li>\n
  4. Anyone have an opinion on SSL session caching<\/li>\n<\/ol>\n
    ################################## MODULES #####################################\r\n\r\n# No additional modules are loaded\r\n################################## NETWORK #####################################\r\n\r\n# My web server is on a different host, so I needed to bind to the public\r\n# network interface. I think we'd *want* to bind to localhost in our\r\n# use case.\r\n# bind 127.0.0.1\r\n# Similarly, I think we'd want 'yes' here\r\nprotected-mode no\r\n\r\n# Might want to use 0 to disable listening on the unsecure port\r\nport 6379\r\ntcp-backlog 511\r\ntimeout 10\r\ntcp-keepalive 300\r\n\r\n################################# TLS\/SSL #####################################\r\n\r\ntls-port 6380\r\ntls-cert-file \/opt\/redis\/ssl\/memcache.pem\r\ntls-key-file \/opt\/redis\/ssl\/memcache.key\r\ntls-ca-cert-dir \/opt\/redis\/ssl\/ca\r\n\r\n# I am not auth'ing clients for simplicity\r\ntls-auth-clients no\r\n\r\ntls-auth-clients optional\r\ntls-protocols \"TLSv1.2 TLSv1.3\"\r\ntls-prefer-server-ciphers yes\r\ntls-session-caching no\r\n\r\n# These would only be set if we were setting up replication \/ clustering\r\n# tls-replication yes\r\n# tls-cluster yes\r\n\r\n################################# GENERAL #####################################\r\n\r\n# This is for docker, we may want to use something like systemd here.\r\ndaemonize no\r\nsupervised no\r\n\r\n#loglevel debug\r\nloglevel notice\r\nlogfile \"\/var\/log\/redis.log\"\r\nsyslog-enabled yes\r\nsyslog-ident redis\r\nsyslog-facility local0\r\n\r\n# 1 might be sufficient -- we *could* partition different apps into different databases\r\n# But I'm thinking, if our keys are basically \"user:target:service\" ... then report_user:RADD:Oracle\r\n# from any web tool would be the same cred. In which case, one database suffices.\r\ndatabases 3\r\n\r\n################################ SNAPSHOTTING ################################\r\n\r\nsave 900 1\r\nsave 300 10\r\nsave 60 10000\r\nstop-writes-on-bgsave-error yes\r\nrdbcompression yes\r\nrdbchecksum yes\r\ndbfilename dump.rdb\r\n\r\n#\r\ndir .\/\r\n\r\n################################## SECURITY ###################################\r\n\r\n# I wasn't setting up any sort of authentication and just using the facts that\r\n# (1) you are on localhost and\r\n# (2) you have the key to decrypt the stuff we stash\r\n# to mean you are authorized.\r\n\r\n############################## MEMORY MANAGEMENT ################################\r\n\r\n# This is what to evict from the dataset when memory is maxed\r\nmaxmemory-policy volatile-lfu\r\n\r\n############################# LAZY FREEING ####################################\r\n\r\nlazyfree-lazy-eviction no\r\nlazyfree-lazy-expire no\r\nlazyfree-lazy-server-del no\r\nreplica-lazy-flush no\r\nlazyfree-lazy-user-del no\r\n\r\n############################ KERNEL OOM CONTROL ##############################\r\n\r\noom-score-adj no\r\n\r\n############################## APPEND ONLY MODE ###############################\r\n\r\nappendonly no\r\nappendfsync everysec\r\nno-appendfsync-on-rewrite no\r\nauto-aof-rewrite-percentage 100\r\nauto-aof-rewrite-min-size 64mb\r\naof-load-truncated yes\r\naof-use-rdb-preamble yes\r\n\r\n############################### ADVANCED CONFIG ###############################\r\n\r\nhash-max-ziplist-entries 512\r\nhash-max-ziplist-value 64\r\nlist-max-ziplist-size -2\r\nlist-compress-depth 0\r\nset-max-intset-entries 512\r\nzset-max-ziplist-entries 128\r\nzset-max-ziplist-value 64\r\nhll-sparse-max-bytes 3000\r\nstream-node-max-bytes 4096\r\nstream-node-max-entries 100\r\nactiverehashing yes\r\nclient-output-buffer-limit normal 0 0 0\r\nclient-output-buffer-limit replica 256mb 64mb 60\r\nclient-output-buffer-limit pubsub 32mb 8mb 60\r\ndynamic-hz yes\r\naof-rewrite-incremental-fsync yes\r\nrdb-save-incremental-fsync yes\r\n\r\n########################### ACTIVE DEFRAGMENTATION #######################\r\n\r\n# Enabled active defragmentation\r\nactivedefrag no\r\n\r\n# Minimum amount of fragmentation waste to start active defrag\r\nactive-defrag-ignore-bytes 100mb\r\n\r\n# Minimum percentage of fragmentation to start active defrag\r\nactive-defrag-threshold-lower 10<\/pre>\n
     <\/p>\n","protected":false},"excerpt":{"rendered":"
    Issue: The multi-step process of retrieving credentials from CyberArk introduce noticeable latency on web tools that utilize multiple passwords. This occurs each execution cycle (scheduled task or user access). Proposal: We will use a redis server to cache credentials retrieved from CyberArk. This will allow quick access of frequently used passwords and reduce latency when …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[33],"tags":[1154,1452,35,427],"class_list":["post-8210","post","type-post","status-publish","format-standard","hentry","category-coding","tag-cyberark","tag-memcached","tag-php","tag-redis"],"_links":{"self":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/8210","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=8210"}],"version-history":[{"count":2,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/8210\/revisions"}],"predecessor-version":[{"id":8213,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/8210\/revisions\/8213"}],"wp:attachment":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=8210"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=8210"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=8210"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}