{"id":9325,"date":"2022-09-15T14:04:03","date_gmt":"2022-09-15T19:04:03","guid":{"rendered":"https:\/\/www.rushworth.us\/lisa\/?p=9325"},"modified":"2022-09-15T14:25:50","modified_gmt":"2022-09-15T19:25:50","slug":"logstash-setting-config-with-environment-variables","status":"publish","type":"post","link":"https:\/\/www.rushworth.us\/lisa\/?p=9325","title":{"rendered":"Logstash – Setting Config with Environment Variables"},"content":{"rendered":"\n

I took over management of an ElasticSearch environment that has a lot<\/em> of configuration inconsistencies. Unfortunately, the previous owners weren’t the ones who built the environment either … so no one knew why<\/em> ServerX did one thing and ServerY did another. Didn’t mess with it (if it’s working, don’t break it!) until we encountered some users who couldn’t find their data — because, depending on which logstash server information transits, stuff ends up in different indices. So now we’re consolidating configurations and I am going to pull the “right” config files into a git repo so we can easily maintain consistency. <\/p>\n\n\n\n

Except … any repository becomes in scope for security scanning. And, really, typing your password in clear text isn’t<\/em> a wonderful plan. So my first step is using environment variables as configuration parameters in logstash. <\/p>\n\n\n\n

The first thing to do is set<\/em> the environment variables somewhere logstash can use them. In my case, I’m using a unit file that sources its environment from \/etc\/default\/logstash<\/p>\n\n\n\n

Once the environment variables are there, enclose the variable name in ${} and use it in the config:<\/p>\n\n\n\n

\"\"<\/a>
Logstash Config<\/figcaption><\/figure>\n\n\n\n

Restart ElasticSearch and verify the pipeline(s) have started successfully. <\/p>\n","protected":false},"excerpt":{"rendered":"

I took over management of an ElasticSearch environment that has a lot of configuration inconsistencies. Unfortunately, the previous owners weren’t the ones who built the environment either … so no one knew why ServerX did one thing and ServerY did another. Didn’t mess with it (if it’s working, don’t break it!) until we encountered some …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1588],"tags":[1590,1589,1643],"class_list":["post-9325","post","type-post","status-publish","format-standard","hentry","category-elk","tag-elasticsearch","tag-elk","tag-logstash"],"_links":{"self":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/9325","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=9325"}],"version-history":[{"count":1,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/9325\/revisions"}],"predecessor-version":[{"id":9327,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/9325\/revisions\/9327"}],"wp:attachment":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=9325"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=9325"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=9325"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}