Linux commands
1 Secret management with temporary variables
NOT TO HAVE passwords in linux command history(.bash_history
:), do this:
Store info into variables:
read -s LDAP_USER read -s LDAP_PASSWORD read -s LDAP_SERVER
check if the variables contain the info
echo "LDAP_USER: $LDAP_USER" echo "LDAP_PASSWORD: $LDAP_PASSWORD" echo "LDAP_SERVER: $LDAP_SERVER"
2 LDAP
# Retrieve entries from administrative databases such as passwd and group getent # list members of a group getent group group_name # check if user belongs to ldap group getent group yg-nas-quant-z | grep "s4X9Xf" # Display the current user ID and group ID, and all associated groups id username # Display all groups a user belongs to groups username # check if multiple users belong to a server getent netgroup LSP7077A | grep -E "S2XXF|S30XX|s7XXc|s8XXf" # check if user belongs to a server getent netgroup servername | grep username
3 Access management
# Show last logged in users last # display bad login attempts of specified user lastb username # shows logged-in users and their activities w cat /etc/sudoers # Edit the sudoers configuration file safely visudo chown chmod # my tip is use setgid on parent directory, that is inherited and then # use setfacl on the group, that would give them the access needed. # attempt with Pitkänen, Arto, but it still does not work setfacl -Rm g:quant:rwX -m d:g:quant:rwX -m m:rwx /[directory of choice]
4 Server information
# display kernel ring buffer messages dmesgn # server uptime uptime # check server ram free -g -h -t # detailed ram info cat /proc/meminfo # same like above just summarized and in GB cat /proc/meminfo | grep -E 'MemTotal|MemFree|MemAvailable|Buffers|Cached' | awk '{print $1 ": " $2/1024/1024 " GB"}' # check server cores lscpu # to see which pkg:s that was updated during the last patch round. I think that # is the best way. good after tsw's rpm -qa -last | more
5 Search
# look for a word "mas" in ALL files(their content) grep -r "mas" . # search for files on the whole server by supressing error messages find / -type f -name "viya-services-status.yml" 2>/dev/null
6 File
# creates a symbolic link(symlink) named "html" in the current directory that points to the "/opt/app/html" directory. ln -s /opt/app/html html # find out how big are the folders du -sh /opt/app/* # show disk usage statistics df -h # estimate file space usage du # size of the folder/file du -sh filename/foldername # see when file was created, modified last time and etc stat filename.txt # cd into dir, shows how much each item in it takes space du -skh * # shows files sudo lsof +L1 # ask for confirmation deleting each folder/file rm -rri /opt/app/R/* # list ALL the folders of the current folder find retail_modelling -type d > test_output.txt # count how many things in the dir ls -la | wc -l # Check when file was created and stuff stat /opt/app/retail_modelling/users/s5743b/fi_kort_rc_ttc.tsv # check top 10 biggest files in home dir du -h /home | sort -rh | head -n 10
7 Processes
fuser -c /opt/app/ ps -ef | grep 2666631 ps -ef | grep rstudio ps aux | grep -i rstudio ps aux | grep -i jupyter kill -9 <processname form the previous command> ps # show running processes ps aux | grep SXXXB | grep rstudio pkill -u sxxxxxf # - NEVEIKIA killall -u sxxxxxf rsession # - NEVEIKIA pkill -9 -u sxxxxxf rsession # - SUVEIKE kill all rstudio sessions sudo killall -9 rserver pgrep rserver # Check if any services remaining systemctl list-units --type=service | grep sas-viya # check all services by name ps aux | grep sas-viya sudo systemctl stop sas-viya-* # kill all services by name
7.1 htop
shortcuts - https://spin.atomicobject.com/2020/02/10/htop-guide/
VIRT
stands for "Virtual Memory." It represents the total virtual
memory used by a process. Virtual memory includes both physical RAM
and swap space on disk.
RES
stands for "Resident Set Size." It represents the portion of a
process's virtual memory that is currently held in physical RAM. In
other words, it shows how much actual physical memory a process is
using.
SHR
stands for "Shared Memory." It represents the portion of the RES
(resident set size) that is shared among multiple processes. This can
be a space-saving feature because shared libraries and code can be
loaded into memory once and used by multiple processes.
8 Network
# show the IP routing table route # find all aliases/DNS of a Linux server that are assigned or being used nslookup <IP address or hostname> # same as above dig -x <IP address> # show you a list of all active network connections and their associated ports netstat -tlnp # find if port 80 is being used netstat -tuln | grep :80 # check what ports are running on the servers nmap -T4 -Pn -p 1-65535 lxxx123a # telnet telnet servername 8301 telnet serername.ad1.net 1433
Labas, cia reiktu pasigilinti i konkreciu serveriu internal firewall nustatymus ir esant poreikiui juos persikonfiguruoti. Panasu, kad dauguma portu ant pvz: xxxx1, xxxx2 serveriu yra isjungti/uzblokuoti. Is networko puses mes galima pagelbet su firewallu taisyklem (srautai tarp skirtingu environmentu, potinkliu), bet vidiniu serveriu portu nustatymu nepriziurim. Jei reikalinga turet connectiona abi sitos vietos turi buti atidarytos (serveris klauso x portu ) + turi buti taisykle firewalle, jei kazkas is 'tolimo' tinklo nori connectintis i serveri tuo x portu.
skirtingi rulsetai ir praleidimai galioja skirtingiems environmentams ir enclave'ams, del to prode ir appvery esantys serveriai nebutinai tures tas pacias taisykles.
9 Logs
journalctl -u jupyterhub -f journalctl -u rstudio-server.service -f
logs to file
ansible-playbook viya-ark-master/playbooks/viya-mmsu/viya-services-status.yml -vvv 2>&1 | tee -a viya_services_status_logs_20230727.txt ansible-playbook viya-ark-master/playbooks/viya-mmsu/viya-services-stop.yml -e "enable_stray_cleanup=true" -vvv 2>&1 | tee -a viya_services_stop_logs_20230727.txt ansible-playbook viya-ark-master/playbooks/viya-mmsu/viya-services-start.yml -vvv 2>&1 | tee -a viya_services_start_logs_20230727.txt # sas installation logs to file ansible-playbook site.yml -vvv 2>&1 | tee -a install_20230809_1500.txt
10 Yum
rpm official repo(yum, RHEL) commands - https://www.rpmfind.net/linux/RPM/ByName.html
# Use yum whatprovides to see what package provides a file. yum whatprovides */ldapsearch # Can see that ldapsearch is part of =openldap-clients= # before installing we can check the content: repoquery -lq openldap-clients # all is good? install yum install openldap-clients yum history yum list installed | grep devel yum remove <reponame> yum erase <reponame> yum repolist all yum repolist enabled yum search 'udunit*' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8 yum search 'R-*' --disablerepo='*' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8 yum info 'R-devel' --disablerepo='*' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8 sudo yum install 'udunits2-devel' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8 --enablerepo=codeready-builder-for-rhel-8-x86_64-rpms # list all available red hat repos yum repolist all # search for package in all repos + EPEL8 repo yum search 'R-*' --disablerepo='*' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8 # search EPEL8 repo yum info 'R-devel' --disablerepo='*' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8 # install from EPEL8 repo sudo yum install 'R-devel' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8 --enablerepo=codeready-builder-for-rhel-8-x86_64-rpms
11 Cron
crontab -l # view your own user's crontab crontab -e # edit your own user's crontab crontab -r # remove your own user's crontab:
12 zip
tar -cvf archive_name.tar files_or_directories_to_archive tar -cvf labas.tar .
13 NAS
# list all drives? cat /etc/fstab # get mount name mount -l |grep sas-store
14 odbc
For setting up ODBC connection you need temp root user. Add temp root user - Model Validation Home - SEB Confluence (sebank.se)
After you have your file you need to create global variable to point to that file location for ODBC to pic up needed information. "export ODBCINI=/etc/odbc.ini" (only for the first time).
nc -vz dbname 1433 ping dbname # You can list the data sources on your system with the following command: odbcinst -q -s # NOTE. You can determine the location of the configuration files on your # system by entering the following command into a terminal: odbcinst -j # Check what ODBC drivers are installed for required DB in "/etc/odbcinst.ini" odbcinst -q -d tsql -S dbname -U username select name from sysdatabases quit
15 Tmux commands
scroll in tmux - ctrl+b+[ then emacs bindings up and down
tmux
- start new
tmux new
-s myname - start new with session name
tmux a #
- (or at, or attach) attach
tmux a -t myname
- attach to named
tmux kill-session -t myname
- kill session
In tmux, hit the prefix ctrl+b and then:
15.1 [A] List all shortcuts
to see all the shortcuts keys in tmux simply use the bind-key ? in my
case that would be CTRL-B ?
15.2 Sessions
:new<CR> new session s list sessions $ name session $ kill session
15.3 Windows (tabs)
c create window w list windows n next window p previous window f find window , name window & kill window
15.4 Panes (splits)
% vertical split " horizontal split o swap panes q show pane numbers x kill pane ; alternate between panes + break pane into window (e.g. to select text by mouse to copy) - restore pane from window space - toggle between layouts <prefix> q (Show pane numbers, when the numbers show up type the key to goto that pane) <prefix> { (Move the current pane left) <prefix> } (Move the current pane right) <prefix> z toggle pane zoom
15.5 Sync Panes
You can do this by switching to the appropriate window, typing your Tmux prefix (commonly Ctrl-B or Ctrl-A) and then a colon to bring up a Tmux command line, and typing:
:setw synchronize-panes
You can optionally add on or off to specify which state you want; otherwise the option is simply toggled. This option is specific to one window, so it won’t change the way your other sessions or windows operate. When you’re done, toggle it off again by repeating the command. tip source
15.6 Resizing Panes
You can also resize panes if you don’t like the layout defaults. I personally rarely need to do this, though it’s handy to know how. Here is the basic syntax to resize panes:
PREFIX : resize-pane -D (Resizes the current pane down) PREFIX : resize-pane -U (Resizes the current pane upward) PREFIX : resize-pane -L (Resizes the current pane left) PREFIX : resize-pane -R (Resizes the current pane right) PREFIX : resize-pane -D 20 (Resizes the current pane down by 20 cells) PREFIX : resize-pane -U 20 (Resizes the current pane upward by 20 cells) PREFIX : resize-pane -L 20 (Resizes the current pane left by 20 cells) PREFIX : resize-pane -R 20 (Resizes the current pane right by 20 cells) PREFIX : resize-pane -t 2 -L 20 (Resizes the pane with the id of 2 left by 20 cells)
15.7 Copy mode:
Pressing PREFIX [ places us in Copy mode. We can then use our movement keys to move our cursor around the screen. By default, the arrow keys work. we set our configuration file to use Vim keys for moving between windows and resizing panes so we wouldn’t have to take our hands off the home row. tmux has a vi mode for working with the buffer as well. To enable it, add this line to .tmux.conf:
setw -g mode-keys vi
With this option set, we can use h, j, k, and l to move around our buffer.
To get out of Copy mode, we just press the ENTER key. Moving around one character at a time isn’t very efficient. Since we enabled vi mode, we can also use some other visible shortcuts to move around the buffer.
For example, we can use "w" to jump to the next word and "b" to jump back one word. And we can use "f", followed by any character, to jump to that character on the same line, and "F" to jump backwards on the line.
Function vi emacs Back to indentation ^ M-m Clear selection Escape C-g Copy selection Enter M-w Cursor down j Down Cursor left h Left Cursor right l Right Cursor to bottom line L Cursor to middle line M M-r Cursor to top line H M-R Cursor up k Up Delete entire line d C-u Delete to end of line D C-k End of line $ C-e Goto line : g Half page down C-d M-Down Half page up C-u M-Up Next page C-f Page down Next word w M-f Paste buffer p C-y Previous page C-b Page up Previous word b M-b Quit mode q Escape Scroll down C-Down or J C-Down Scroll up C-Up or K C-Up Search again n n Search backward ? C-r Search forward / C-s Start of line 0 C-a Start selection Space C-Space Transpose chars C-t
15.8 Misc
d detach t big clock ? list shortcuts : prompt
15.9 Configurations Options:
# Mouse support - set to on if you want to use the mouse * setw -g mode-mouse off * set -g mouse-select-pane off * set -g mouse-resize-pane off * set -g mouse-select-window off # Set the default terminal mode to 256color mode set -g default-terminal "screen-256color" # enable activity alerts setw -g monitor-activity on set -g visual-activity on # Center the window list set -g status-justify centre # Maximize and restore a pane unbind Up bind Up new-window -d -n tmp \; swap-pane -s tmp.1 \; select-window -t tmp unbind Down bind Down last-window \; swap-pane -s tmp.1 \; kill-window -t tmp
16 vim
:set paste :set nopaste :set nonu :set nu # no highlighting: :noh
17 ansible
ansible-galaxy collection list # install default version ansible-galaxy collection install community.general # install specific version: ansible-galaxy collection install geerlingguy.apache,v1.0.0 7.0.1
to delete a package - delete it's folder here:
for example I deleted community.general by doing rm -rf community_old/ in home/instaluser.ansible/collections/ansible_collections