set up key exchanges<\/a> on your own ID too – even from your Windows desktop – and avoid typing passwords.<\/li>\n<\/ul>\nOnce you are logged in, you can start a screen session. Normally, anything you are running is terminated if your SSH session terminates (e.g. if you use Cygwin or Putty to connect to a box from your laptop that is VPN\u2019d into the network & your VPN drops \u2026 everything you were doing in the SSH session is terminated.). You can use screen to set up a persistent session \u2013 you can reconnect to that session should your SSH connection get interrupted, other people can connect to the session to monitor a long running script, or multiple people can connect to the session and all see the same thing (screen sharing).<\/p>\n
To start a new screen session, screen -S SessionName <\/em><\/strong>where SessionName is something that identifies the screen session as yours (e.g. LJRPasswordResync was the session I used when resyncing all employee and contractor passwords for OIDM \u2013 this includes both my initials and the function I\u2019m running in the session). To see the currently running sessions, use screen \u2013ls<\/strong><\/p>\n[lisa@server810 ~]# screen -ls<\/p>\n
There is a screen on:<\/p>\n
8210.LJR\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 (Detached)<\/p>\n
1 Socket in \/tmp\/screens\/S-lisa.<\/p>\n
The output contains both a session ID number (green) and a session name (blue) separated by a full stop. You can use either to connect to a screen session (the name is case sensitive!). To reconnect, use screen \u2013x SessionName<\/em><\/strong> or screen \u2013x SessionID<\/em><\/strong><\/p>\nTo determine if you are currently\u00a0in<\/em> a screen session, look at the upper left hand corner of your Putty window. The title will change to include screen when you are in a screen session. Alternately echo the STY environment variable. If you get nothing, it is not a screen session. If you get output, it is the PID and name of your current screen session.<\/p>\n[lisa@server810 ~]# echo $STY
\n43116.LJR<\/p>\n
SUDO \u2013 <\/strong>The sudo command lets you execute commands that your ID is not normally privileged to run. There is configuration to sudo (maintained by ITSecurity) that defines what<\/em> you can run through sudo. If, for example, you are unable to edit a file but are<\/em> permitted to sudo vim \u2026 editing a file using \u201cvi \/path\/to\/file.xtn\u201d will throw an error if you attempt to save changes, but running \u201csudo vi \/path\/to\/file.xtn\u201d would allow you to save changes to the file.<\/p>\nSubstitute user \u2013 <\/strong>The command su lets you substitute a uidnumber for yours \u2013 this means you become<\/em> that user.<\/p>\nCombining SUDO and SU \u2013 <\/strong>Once we are logged into LX810 with our user ID, we can use sudo su \u2013 root<\/strong> to become root without actually knowing the root password. The \u201cspace dash space\u201d in the su command means the user\u2019s environment is loaded. If you omit the space dash space, you\u2019ll still be logged in as the root user, but your environment will be left in place.<\/p>\nGenerally speaking, allowing sudo to root is a bad idea (i.e. don’t do this even though you’ll see it on a lot of our old servers). This is because root has full access to everything and running the shell as root is insecure and typos can be disastrous.<\/p>\n
Navigating \u2013<\/strong> You are in a DOS-like command line interface. The interface is known as a shell \u2013 root on LX810 is a bash shell. The default for a CUSO ID is the korn shell (\/bin\/ksh) \u2013 you can change your shell in your LDAP account to \/bin\/bash (or \/bin\/csh for the C shell) and subsequent logons will use the new shell. You can try each one and see which you prefer, you can use korn because it is the default from CUSO, or you can use bash because it matches the instructions I write.<\/p>\nFrom a file system navigation perspective, you will be in the logon user\u2019s home directory. If you aren\u2019t sure where you are in the file system, type pwd<\/strong> and the present working directory will be output.<\/p>\nTo see what is in a directory, use ls<\/strong> \u2026 there are additional parameters you can pass (in Linux parameters are passed with a dash or two dashes). Adding -a lists *all* files (including the hidden ones, any file where the name starts with a full stop is a hidden file). Adding -l provides a long listing (file owners, sizes, modified dates). Adding -h lists file sizes in human readable format. You can pass each parameter separately (ls \u2013a \u2013l \u2013h) or by concatenating them together (ls \u2013alh)<\/p>\nYou can use wc to count the number of lines either in a file (wc \u2013l \/path\/to\/file.xtn) or the output of ls (ls \u2013al | wc \u2013l) \u2013 this is useful on our sendmail servers when you have received a queue length alert and done something to clear out some of the queue. In sendmail particularly, there are two files for each message so you need to divide the line count by 2.<\/p>\n
To change to a different directory, use cd<\/strong> \u2013 e.g. cd \/etc\/mail<\/em> will change the working directory to \/etc\/mail.<\/p>\nTo delete a file, use rm<\/strong> \/path\/to\/file.xtn<\/strong> \u2013 this is the safe way to run it, it will prompt for confirmation for each file being deleted. You can use wildcards (rm \/path\/to\/files*) to delete multiple files. You can add a -f parameter to not<\/em> be prompted \u2013 which is more dangerous as you may have typed the wrong thing and it\u2019ll be deleted without prompting. You can add a \u2013r parameter for recursive (get rid of everything under a path). Not too dangerous as long as you have the prompt coming up \u2013 but if you use \u2013r in conjunction with \u2013f (rm \u2013rf) \u2026 you can do a lot of damage. Absolute worst case would be recursive force delete from \/ \u2026 which would mean every file on disk goes away. Don\u2019t do that J<\/p>\nIf you are not sure where a file you need is located, you can use either find or locate. The locate command is not always installed, so you would need<\/em> to use the find command. Locate uses an index database \u2013 so it\u2019s quicker, but it doesn\u2019t know about files created\/deleted since the index was updated.<\/p>\nTo use locate, use locate -i filename <\/strong>where filename is some part of the filename. The -i performs a case insensitive search \u2013 if you know<\/em> the proper casing, you do not need to include this parameter.<\/p>\nTo use find, you need to indicate the root of the search (if you have no clue, use \u2018\/\u2019 which is the top level directory) as well as the exact file name that you want (not a substring of the file name like locate will let you do). Finding a file named audit.log that is somewhere on the disk would be find \/ -name audit.log<\/strong><\/p>\nCustomizing shell environment \u2013 <\/strong>You can customize your shell environment. The system-wide shell environment settings are in \/etc and are specific to the shell. For a bash shell, it is \/etc\/bashrc<\/p>\nIndividual user settings are in a hidden file within their home directory. For the bash shell, the user specific settings are in $HOME\/.bashrc ($HOME is a variable for the current logon user\u2019s home directory).<\/p>\n
For a shared account, adding things to $HOME\/.bashrc isn\u2019t the best idea \u2013 your preferred settings can differ from someone else\u2019s preferences. We make our own rc file in $HOME for the shared account (I actually set my .bashrc as world-readable and linked<\/em> the shared ID $HOME\/.ljlrc to my personal .bashrc file so I only have to remember to edit one file). You can load your personal preferences using source $HOME\/.yourrc <\/strong>or you can load someone else\u2019s preferences by sourcing their<\/em> file in the shared account\u2019s home directory (source $HOME\/.ljlrc will load in mine).<\/p>\nService Control \u2013 <\/strong>Most of our Linux systems still use systemd (init.<\/strong>d scripts) to start and stop services. You can find the scripts in \/etc\/init.d \u2013 these are readable text scripts. All scripts will have a start and stop command, and many have restart and status as additional commands. To control a service, you can use service servicename command, \/sbin\/service servicename command<\/strong> or \/etc\/init.d\/servicename command<\/strong> \u2013 same thing either way. If you are controlling the service through sudo, though, you need to use the technique that is permitted to your UID in the sudo configuration.<\/p>\nIf you use a command that isn\u2019t implemented in the script, you will get usage information. You can use a semicolon to chain commands (like the & operator in DOS) \u2013 so \/etc\/init.d\/sendmail restart<\/strong> is the same thing as running \/etc\/init.d\/sendmail stop;\/etc\/init.d\/sendmail start<\/strong><\/p>\nProcess utilization \u2013 <\/strong>To see what the processor and memory utilization is like on a box (as well as which processes are causing this utilization), use top<\/strong>. When top has launched, the first few lines give you the overall usage. The load average (blue below) tells you the load during the last one, five, and fifteen minutes \u2013 1.00 is 100% on a single core system, 2.00 is 100% on a two core system, etc. Over the 100% number for a system means stuff got queued waiting for CPU cycles to become available.<\/p>\nThe current CPU utilization (green below) breaks out usage by user tasks, system tasks, nice\u2019d processes (generally nothing here), idle, io wait, hardware irq, software irq.<\/p>\n
The memory usage (red below) shows used and free memory.<\/p>\n
top – 13:58:30 up 486 days, \u00a02:16,\u00a0 9 users,\u00a0 load average: 0.34, 0.24, 0.25<\/p>\n
Tasks: 162 total,\u00a0\u00a0 1 running, 161 sleeping,\u00a0\u00a0 0 stopped,\u00a0\u00a0 0 zombie<\/p>\n
Cpu(s):\u00a0 0.4% us,\u00a0 0.1% sy,\u00a0 0.0% ni, 99.5% id,\u00a0 0.0% wa,\u00a0 0.0% hi,\u00a0 0.0% si<\/p>\n
Mem:\u00a0\u00a0 4147208k total,\u00a0 2107876k used,\u00a0 2039332k free,\u00a0\u00a0\u00a0 62372k buffers<\/p>\n
Swap:\u00a0 2064376k total,\u00a0\u00a0\u00a0\u00a0 1352k used,\u00a0 2063024k free,\u00a0 1167652k cached<\/p>\n
<\/p>\n
The process list can be sorted by whatever you need \u2013 if the box is CPU-bound, type an upper case C to sort by CPU usage. If it is memory bound, type an upper case M to sort by memory usage.<\/p>\n
PID USER\u00a0\u00a0\u00a0\u00a0\u00a0 PR\u00a0 NI %CPU\u00a0\u00a0\u00a0 TIME+\u00a0 %MEM\u00a0 VIRT\u00a0 RES\u00a0 SHR S COMMAND<\/p>\n
23190 root\u00a0\u00a0\u00a0\u00a0\u00a0 15\u00a0\u00a0 0\u00a0\u00a0\u00a0 1\u00a0\u00a0 5:43.81 14.9\u00a0 608m 605m 2872 S perl<\/p>\n
14225 root\u00a0\u00a0\u00a0\u00a0\u00a0 16\u00a0\u00a0 0\u00a0\u00a0\u00a0 0\u00a0\u00a0 7:14.20\u00a0 1.7\u00a0 170m\u00a0 69m\u00a0 60m S cvd<\/p>\n
14226 root\u00a0\u00a0\u00a0\u00a0\u00a0 16\u00a0\u00a0 0\u00a0\u00a0\u00a0 0\u00a0\u00a0 1:30.32\u00a0 1.4\u00a0 147m\u00a0 57m\u00a0 50m S EvMgrC<\/p>\n
4585 root\u00a0\u00a0\u00a0\u00a0\u00a0 16\u00a0\u00a0 0\u00a0\u00a0\u00a0 0 212:01.99\u00a0 1.1\u00a0 230m\u00a0 43m 6368 S dsm_om_connsvc3<\/p>\n
4003 root\u00a0\u00a0\u00a0\u00a0\u00a0 16\u00a0\u00a0 0\u00a0\u00a0\u00a0 0\u00a0\u00a0 2729:44\u00a0 0.6\u00a0 171m\u00a0 24m 3364 S dsm_sa_datamgr3<\/p>\n
24552 root\u00a0\u00a0\u00a0\u00a0\u00a0 16\u00a0\u00a0 0\u00a0\u00a0 13\u00a0\u00a0 0:36.16\u00a0 0.3 17804\u00a0 12m 2900 S perl<\/p>\n
<\/p>\n
The first column shows the PID (process ID). Some commands as listed in top are obvious what they actually are (httpd is the apache web server, for instance) and others aren\u2019t (perl, above, doesn\u2019t really tell us *what* is using the CPU). To determine what the PID actually is, use ps \u2013efww | grep PID#<\/em><\/strong><\/p>\n[lisa@server810 Sendmail-CheckQSize]# ps -efww | grep 23190<\/strong><\/p>\nroot\u00a0\u00a0\u00a0\u00a0 23190 23187\u00a0 0 01:23 ?\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 00:05:44 \/usr\/bin\/perl \/home\/NDSSupport\/Scripts\/osrOCSProvisioning\/_syncIMEnabledFromCSO.pl<\/p>\n
root\u00a0\u00a0\u00a0\u00a0 24645 16640\u00a0 0 14:10 pts\/10\u00a0\u00a0 00:00:00 grep 23190<\/p>\n
<\/p>\n
You will see the full command that is running \u2013 in this case a particular perl script. Note that you may also find your grep command in the list \u2026 depends a bit on timing if it shows up or not.<\/p>\n
You may need to restart a service to clear something that has a memory leak. You may need to stop the process outside of the service control (e.g. stopping the sendmail service doesn\u2019t shut down all current threads). To stop a process, use kill PID#<\/strong> \u2026 this is basically asking a process nicely to stop. It will clean up its resources and shut down cleanly. use ps \u2013efww to see if the process is still running. If it still is, use kill -9 PID#<\/strong> which is not<\/em> asking nicely. Most things to which a process is connected will clean up their own resources after some period of client inactivity (i.e. you aren\u2019t causing a huge number of problems for someone else by doing this) but it is cleaner to use kill without the \u201cdo it NOW!!!\u201d option first.<\/p>\nTail and Grep<\/strong> \u2013 Tail is a command that outputs the last n<\/em> lines of a file. It has a parameter that outputs new lines as they get appended to the file. On *n?x systems, you can use tail \u2013F<\/strong> \/path\/to\/file.xtn<\/strong> and lines will be output as they show up. This is particularly useful on log files where the system is continually adding new info at the bottom of the file. We put Windows ports of these utilities on our Windows servers \u2013 but the Windows port of tail does not support \u2013F (there\u2019s a good reason that has to do with the difference between Unix-like and Windows file systems). You can use tail \u2013f<\/strong> instead \u2013 if the log file rolls (gets moved to another file and a new file is started) you won\u2019t continue to get output like you will with \u2013F \u2026 but you can ctrl-c to terminate the tail & start it again to start seeing the new lines again.<\/p>\nGrep is a command line search program. You can use grep to find lines in a file containing a string (or regex pattern, but learning regex is a question for LMGTFY.com) \u2013 to find all of the mail addressed to or from me in a sendmail log, grep \u2013i rushworth \/var\/log\/maillog<\/strong> \u2013 the dash i means case insensitive search.<\/p>\nGrep will also search piped input instead of a file \u2013 this means you can send the output<\/em> of tail to grep and display only the lines matching the pattern for which you search.<\/p>\ntail -f \/var\/log\/maillog | grep \u2013i rushworth <\/strong>will output new lines of the maillog as they come in, but only display the ones with my name.<\/p>\nVIM<\/strong> \u2013 The non-visual text editor is vim \u2013 which is usually invoked using \u2018vi\u2019, but vi is an actual program that is like but not exactly the same as vim (vim is improved vi). The vim installation contains a very nice tutorial \u2013 invoked by running vimtutor<\/strong><\/p>\nVIM has both a command mode and an editing mode. When in command mode, different keys on the keyboard have different functions. There are \u201cquick reference\u201d guides and \u201ccheat sheets\u201d online for vim \u2013 most people I know have a quick ref guide or cheat sheet taped next to their computer for quite some time before vim commands become well known.<\/p>\n
History<\/strong> \u2013 Linux maintains a history of commands run in a session. This spans logons (you\u2019ll see commands run last week even through you\u2019ve logged on and off six times between then) but when there are multiple sessions for the same user, there can be multiple history files. Which is all a way of saying you may not see something you expect to see, or you may see things you don\u2019t expect. The output of history<\/strong> shows the command history for the current logon session. You can pipe the output to grep and find commands in the history \u2013 for example, if you don\u2019t remember how to start a service, you can use history | grep start<\/strong> and get all commands that contain the string start<\/p>\n[lisa@server855 ~]# history | grep start<\/p>\n
7\u00a0 service ibmslapd start<\/p>\n
15\u00a0 service ibmslapd restart<\/p>\n
42\u00a0 service ibmslapd start<\/p>\n
56\u00a0 service ibmslapd restart<\/p>\n
71\u00a0 service ibmslapd start<\/p>\n
95\u00a0 service ibmslapd start<\/p>\n
107\u00a0 service ibmslapd start<\/p>\n
115\u00a0 service ibmslapd restart<\/p>\n
289\u00a0 service ibmslapd start<\/p>\n
303\u00a0 service ibmslapd start<\/p>\n
408\u00a0 service ibmslapd start<\/p>\n
419\u00a0 service ibmslapd start<\/p>\n
430\u00a0 service ibmslapd start<\/p>\n
443\u00a0 service ibmslapd start<\/p>\n
If a command fails, it will still be in the history (all of my typo\u2019s are in there!), but if you see the same command a number of times \u2026 it\u2019s probably correct. You can copy\/paste the command if you need to edit it to run (or even to run it as-is). You can run the exact command again by typing bang followed by the line number of the history output (!115<\/strong> with the history above would re-run \u201cservice ibmslapd restart\u201d).<\/p>\nSymbolic Links<\/strong><\/p>\nLinux symbolic links are nothing like Windows shortcuts, although I see people saying that. Shortcuts are independent files that contain a path to the referenced file. Linux sym links are just pointers to the inode for the file. They\u00a0are<\/em> the file, just allowing it to be used in a different location. This is a bit like memory addressing in programming — anything that reads from the memory address will get the same data, and anything that writes to the memory address. When you do a long list (ls -al or just ll), you will see both the file name and the file to which it points:<\/p>\nlrwxrwxrwx 1 root root 19 Aug 17 13:54 ljrtest -> \/tmp\/dnsexit-ip.txt<\/p>\n
The “l” at the start of the line indicates that it is a link.<\/p>\n","protected":false},"excerpt":{"rendered":"
We’ve got a few new people at work who don’t have any Linux experience, and I was asked to do a quick crash course on some super fundamental logging in \/ navigating \/ restarting service stuff so their first on call rotation wouldn’t be quite so stressful. Publishing the overview here in case it is …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30],"tags":[369,294,370,371,368],"class_list":["post-1489","post","type-post","status-publish","format-standard","hentry","category-system-administration","tag-how-to","tag-linux","tag-linux-intro","tag-linux-introduction","tag-tutorial"],"_links":{"self":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/1489","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1489"}],"version-history":[{"count":4,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/1489\/revisions"}],"predecessor-version":[{"id":1793,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/1489\/revisions\/1793"}],"wp:attachment":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1489"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1489"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1489"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}