Month: October 2020

On Patriotic History

History is written by the victor. They can tell us how nice they were (or at least how necessary their not-niceness was). But the fact those who win get to write history in their favor doesn’t negate the value of ensuring people have a more robust view of what actually transpired. The good and the bad. Which makes Trump’s idea of a more patriotic history quite frightening.
 
In software development, we have “retrospectives” — a meeting where everyone chats about how the last project went. What worked well. What didn’t work well. It’s not meant to be subversive, negative, or blamey — it’s meant to get people thinking about how we could improve the things that didn’t work well. And to feel proud about the things that did go well. I’d love to see this approach taken to teaching history.
 
By focusing only on the good aspects, you lose important information. A tangentially related example: my daughter’s social studies book attempts to cover the concept of savings and loans. They talked about saving money to buy something bigger later and about the bank giving you money to buy something bigger *now* and you you give the money back later. And omitted the entire concept of interest. Elementary schools are telling kids that the bank will give you a couple hundred k to buy a house, you pay them back over time, and it’s all beautiful. I pulled up my credit card statement and showed her how the grand we spent last month could be paid back immediately — the bank gave me a grand, I paid them a grand back, and they gave me 30$ in bonus cash back for using their service — but that’s not a sustainable business model. How does the bank pay for the building downtown? The people who work there? The advertising? The computer systems? I showed her the “if you pay the minimum” and “if you pay more than the minimum, look how much you ‘save'” box where that grand could cost me three grand. Or I could ‘save’ 1500 by paying more than the minimum due.

Tar Excluding Git Folders

You can, of course, use –exclude and avoid adding the .git folders to your tar archive, but I discovered a really cool option that excludes the folders created by a whole host of version control systems:

--exclude-vcs

Which, as of version 1.32, means excluding CVS, RCS, SCCS, git, SVN, Arch, Bazaar, Mercurial, and Darcs as follows:

  • CVS/ — recursive
  • RCS/ — recursive
  • SCCS/ — recursive
  • .git/ — recursive
  • .gitignore
  • .gitmodules
  • .gitattributes
  • .cvsignore
  • .svn/ — recursive
  • .arch-ids/ — recursive
  • {arch}/ — recursive
  • =RELEASE-ID
  • =meta-update
  • =update
  • .bzr
  • .bzrignore
  • .bzrtags
  • .hg
  • .hgignore
  • .hgrags
  • _darcs

Cyberark — Error Listing Accounts

I was getting an odd error from my attempt to list accounts in Cyberark — “Object reference not set to an instance of an object”. Searching the Internet yielded a lot of issues that weren’t my problem (ampersands in account names in an older version, issues with SSL {and, seriously, someone says disable SSL on the connection they use to retrieve passwords!?! And not just random someone, but RAND?!?}). My issue turned out to be that I was copy/pasting code and used requests.post instead of requests.get — attempting to POST to a GET URL generates this error too.

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): cyberark.example.com:443
DEBUG:urllib3.connectionpool:https://cyberark.example.com:443 “POST /PasswordVault/API/auth/Cyberark/Logon HTTP/1.1” 200 182
Before request, header is {‘Content-Type’: ‘application/json’, ‘Authorization’: ‘5TQz5WVjYm5tMjBh5C00M5YyLT50MjYt5Tc2Y5I2ZDI5…AwMDA5MDA7’}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): cyberark.example.com:443
DEBUG:urllib3.connectionpool:https://cyberark.example.com:443 “POST /PasswordVault/api/Accounts?search=sample_account&searchType=contains HTTP/1.1” 500 97
{“ErrorCode”:”CAWS00001E”,”ErrorMessage”:”Object reference not set to an instance of an object.”} 500 Internal Server Error

DigiMash Unboxing

Our beer-brewing equipment arrived today! The packaging was not robust — every box had some fairly substantial damage. There’s a dent inside the kettle and one of the fermenters has a pushed in section (supposedly this will pop right out when we fill it). My first surprise was that we got a DigiBoil in a box and the mash upgrade kit in another box. The kettle is even branded as DigiBoil. Which … from a manufacturing / material standpoint makes sense. Why would they have two different sets of packaging and product labels? And three different SKU’s — DigiBoil, DigiMash kit, DigiBoil Mash Upgrade. It was a better deal buying the DigiBoil package — the DigiMash 65L was $259.99 and the mash upgrade $89.99, so we saved ten bucks ordering

Microsoft Whiteboard Sticky Notes and Text Box

Two ways to add text to Microsoft Whiteboard sessions — ways that aren’t dragging your finger or mouse around in an attempt to draw legible text — are available. I’d like to be able to change the font in the text box — I get that their font choice is meant to evoke hand-written text, but it strikes me as non-professional.

PrivateTmp Strangeness — apachectl v/s systemctl

This is a very strange problem — we had a web server upgraded recently. We use “sudo apachectl start” to bring up the server since the server is maintained by a dedicated Unix support team, and the site worked fine. Until … Sunday morning after the log rotation. Then Box Spout was unable to access the XML data for an Excel file to compress it. Lots of errors:

 

[Mon Sep 14 10:59:39.137728 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: ZipArchive::close(): Zlib error: stream error in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 199, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.137785 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: fopen(/tmp/xlsx5f5f934699dcd9.17695276.zip): failed to open stream: No such file or directory in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 213, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.137803 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: stream_copy_to_stream() expects parameter 1 to be resource, boolean given in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 214, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.137814 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: fclose() expects parameter 1 to be resource, boolean given in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 215, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.138360 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Fatal error: Uncaught exception ‘Box\\Spout\\Common\\Exception\\IOException’ with message ‘Cannot perform I/O operation outside of the base folder: /tmp’ in /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php:130\nStack trace:\n#0 /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php(82): Box\\Spout\\Common\\Helper\\FileSystemHelper->throwIfOperationNotInBaseFolder(‘/tmp/xlsx5f5f93…’)\n#1 /path/to/site/classes/vendor/box/spout/src/Spout/Writer/XLSX/Helper/FileSystemHelper.php(369): Box\\Spout\\Common\\Helper\\FileSystemHelper->deleteFile(‘/tmp/xlsx5f5f93…’)\n#2 /path/to/site/classes/vendor/box/spout/src/Spout/Writer/XLSX/Internal/Workbook.php(134): Box\\Spout\\Writer\\XLSX\\Helper\\FileSystemHelper->zipRootFolderAndCopyToStream(Resource id #26)\n#3 /path/to/site/ in /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php on line 130, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.139468 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: ZipArchive::close(): Zlib error: stream error in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 199, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.139504 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: fopen(/tmp/xlsx5f5f934699dcd9.17695276.zip): failed to open stream: No such file or directory in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 213, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.139515 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: stream_copy_to_stream() expects parameter 1 to be resource, boolean given in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 214, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.139533 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: fclose() expects parameter 1 to be resource, boolean given in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 215, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.139599 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Fatal error: Uncaught exception ‘Box\\Spout\\Common\\Exception\\IOException’ with message ‘Cannot perform I/O operation outside of the base folder: /tmp’ in /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php:130\nStack trace:\n#0 /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php(82): Box\\Spout\\Common\\Helper\\FileSystemHelper->throwIfOperationNotInBaseFolder(‘/tmp/xlsx5f5f93…’)\n#1 /path/to/site/classes/vendor/box/spout/src/Spout/Writer/XLSX/Helper/FileSystemHelper.php(369): Box\\Spout\\Common\\Helper\\FileSystemHelper->deleteFile(‘/tmp/xlsx5f5f93…’)\n#2 /path/to/site/classes/vendor/box/spout/src/Spout/Writer/XLSX/Internal/Workbook.php(134): Box\\Spout\\Writer\\XLSX\\Helper\\FileSystemHelper->zipRootFolderAndCopyToStream(Resource id #26)\n#3 /path/to/site in /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php on line 130, referer: https://hostname.example.com/path/to/code.php

 

The postupdate script is “systemctl reload httpd.service” — so not exactly the same thing we used to launch the service originally. But I’ve never seen differing behavior between apachectl and systemctl started HTTPD instances. Quick/dirty solution is to disable PrivateTmp, but I’m hoping to be able to isolate why exactly the postupdate script appears to break the service’s access to the private tmp space.

Python Time-Expiring Cache

I needed to post information into a SharePoint Online list. There’s an auth header required to post data, but the authentication expires every couple of minutes. Obviously, I could just get a new auth string each time … but that’s terribly inefficient. I could also use what I had and refresh it when it fails … inelegant but effective. I wanted, instead, to have a value cached for a duration slightly less than the server-side expiry on the auth string. This decorator allows me to use the cached auth header string for some period of time and actually run the function that grabs the auth header string before the string is invalidated.

import functools
import time
from datetime import datetime

def timed_cache(iMaxAge, iMaxSize=128, boolTyped=False):
    #######################################
    # LRU cache decorator with expiry time
    #
    # Args:
    #    iMaxAge: Seconds to live for cached results. 
    #    iMaxSize: Maximum cache size (see functools.lru_cache).
    #    boolTyped: Cache on distinct input types (see functools.lru_cache).
    #######################################
    def _decorator(fn):
        @functools.lru_cache(maxsize=iMaxSize, typed=boolTyped)
        def _new(*args, __time_salt, **kwargs):
            return fn(*args, **kwargs)

        @functools.wraps(fn)
        def _wrapped(*args, **kwargs):
            return _new(*args, **kwargs, __time_salt=int(time.time() / iMaxAge))
        return _wrapped

    return _decorator

# Usage example -- 23 second cache expiry
@timed_cache(23)
def slow_function(iSleepTime: int):
    datetimeStart = datetime.now()
    time.sleep(iSleepTime)
    return f"test started at {datetimeStart} and ended at at {datetime.now()}"


print(f"Start at {datetime.now()}")

for i in range(1, 50):
    print(slow_function(5))
    
    if i % 5 is 0:
        print(f"sleeping 5 at {datetime.now()}")
        time.sleep(5)
        print(f"done sleeping at {datetime.now()}\n\n")

print(f"Ended at at {datetime.now()}")

Ohio Absentee Ballot — Confirming Rejected Ballot

I requested an absentee ballot this year — we’d used absentee ballots once back when we lived in Geauga county and delivered the ballots to the Board of Elections office, but we generally vote in person. I wanted Anya to experience the process (and they give out cute stickers!). But we’ll have more time to explore what the ballot actually is at home, and there won’t be a bunch of people about. But — and the one thing that makes me want to do the early in-person voting — they can reject your ballot if the signature doesn’t match. I didn’t have a problem with my absentee ballot request, but Scott’s got rejected for a signature mismatch. And that had be worried about my actual ballot.

But, amid all of Trump’s blathering last night? Biden managed to convey that a lot of states have a process for curing rejected absentee ballots. I had no idea. Per Ohio Rev Code § 3509.06:

“(b) If the election officials find that the identification envelope statement of voter is incomplete or that the information contained in that statement does not conform to the information contained in the statewide voter registration database concerning the voter, the election officials shall mail a written notice to the voter, informing the voter of the nature of the defect. The notice shall inform the voter that in order for the voter’s ballot to be counted, the voter must provide the necessary information to the board of elections in writing and on a form prescribed by the secretary of state not later than the seventh day after the day of the election. The voter may deliver the form to the office of the board in person or by mail. If the voter provides the necessary information to the board of elections not later than the seventh day after the day of the election and the ballot is not successfully challenged on another basis, the voter’s ballot shall be processed and counted in accordance with this section.”

Which means I’d be notified if my ballot is rejected, and I could go to the Board of Elections within seven days of the election to provide whatever sort of identification they need to be satisfied. A total of fifteen states provide a remediation path for signature challenges — details for those states at https://www.ncsl.org/research/elections-and-campaigns/vopp-table-15-states-that-permit-voters-to-correct-signature-discrepancies.aspx