PROJECTS NOTES HOME

Linux commands

1 Secret management with temporary variables

NOT TO HAVE passwords in linux command history(.bash_history:), do this:

Store info into variables:

read -s LDAP_USER
read -s LDAP_PASSWORD
read -s LDAP_SERVER

check if the variables contain the info

echo "LDAP_USER: $LDAP_USER"
echo "LDAP_PASSWORD: $LDAP_PASSWORD"
echo "LDAP_SERVER: $LDAP_SERVER"

2 LDAP

# Retrieve entries from administrative databases such as passwd and group
getent
# list members of a group
getent group group_name
# check if user belongs to ldap group
getent group yg-nas-quant-z | grep "s4X9Xf"
# Display the current user ID and group ID, and all associated groups
id username
# Display all groups a user belongs to
groups username
# check if multiple users belong to a server
getent netgroup LSP7077A | grep -E "S2XXF|S30XX|s7XXc|s8XXf"
# check if user belongs to a server
getent netgroup servername | grep username

3 Access management

# Show last logged in users
last
# display bad login attempts of specified user
lastb username
# shows logged-in users and their activities
w

cat /etc/sudoers

# Edit the sudoers configuration file safely
visudo

chown
chmod

# my tip is use setgid on parent directory, that is inherited and then
# use setfacl on the group, that would give them the access needed.
# attempt with Pitkänen, Arto, but it still does not work

setfacl -Rm g:quant:rwX -m d:g:quant:rwX -m m:rwx /[directory of choice]

4 Server information

# display kernel ring buffer messages
dmesgn
# server uptime
uptime
# check server ram
free -g -h -t
# detailed ram info
cat /proc/meminfo
# same like above just summarized and in GB
cat /proc/meminfo | grep -E 'MemTotal|MemFree|MemAvailable|Buffers|Cached' | awk '{print $1 ": " $2/1024/1024 " GB"}'
# check server cores
lscpu
# to see which pkg:s that was updated during the last patch round. I think that
# is the best way. good after tsw's
rpm -qa -last | more

5 Search

# look for a word "mas" in ALL files(their content)
grep -r "mas" .
# search for files on the whole server by supressing error messages
find / -type f -name "viya-services-status.yml" 2>/dev/null

6 File

# creates a symbolic link(symlink) named "html" in the current directory that points to the "/opt/app/html" directory.
ln -s /opt/app/html html
# find out how big are the folders
du -sh /opt/app/*
# show disk usage statistics
df -h
# estimate file space usage
du
# size of the folder/file
du -sh filename/foldername
# see when file was created, modified last time and etc
stat filename.txt
# cd into dir, shows how much each item in it takes space
du -skh *
# shows files
sudo lsof +L1
# ask for confirmation deleting each folder/file
rm -rri /opt/app/R/*
# list ALL the folders of the current folder
find retail_modelling -type d > test_output.txt
# count how many things in the dir
ls -la | wc -l
# Check when file was created and stuff
stat /opt/app/retail_modelling/users/s5743b/fi_kort_rc_ttc.tsv
# check top 10 biggest files in home dir
du -h /home | sort -rh | head -n 10

7 Processes

fuser -c /opt/app/

ps -ef | grep 2666631

ps -ef | grep rstudio

ps aux | grep -i rstudio

ps aux | grep -i jupyter

kill -9 <processname form the previous command>

ps                              # show running processes

ps aux | grep SXXXB | grep rstudio

pkill -u sxxxxxf              # - NEVEIKIA
killall -u sxxxxxf rsession   # - NEVEIKIA
pkill -9 -u sxxxxxf rsession  # - SUVEIKE

kill all rstudio sessions
sudo killall -9 rserver

pgrep rserver                   # Check if any services remaining

systemctl list-units --type=service | grep sas-viya # check all services by name
ps aux | grep sas-viya

sudo systemctl stop sas-viya-*  # kill all services by name

7.1 htop

shortcuts - https://spin.atomicobject.com/2020/02/10/htop-guide/

VIRT stands for "Virtual Memory." It represents the total virtual memory used by a process. Virtual memory includes both physical RAM and swap space on disk.

RES stands for "Resident Set Size." It represents the portion of a process's virtual memory that is currently held in physical RAM. In other words, it shows how much actual physical memory a process is using.

SHR stands for "Shared Memory." It represents the portion of the RES (resident set size) that is shared among multiple processes. This can be a space-saving feature because shared libraries and code can be loaded into memory once and used by multiple processes.

8 Network

# show the IP routing table
route
# find all aliases/DNS of a Linux server that are assigned or being used
nslookup <IP address or hostname>
# same as above
dig -x <IP address>
# show you a list of all active network connections and their associated ports
netstat -tlnp

# find if port 80 is being used
netstat -tuln | grep :80
# check what ports are running on the servers
nmap -T4 -Pn -p 1-65535 lxxx123a
# telnet
telnet servername 8301
telnet serername.ad1.net 1433

Labas, cia reiktu pasigilinti i konkreciu serveriu internal firewall nustatymus ir esant poreikiui juos persikonfiguruoti. Panasu, kad dauguma portu ant pvz: xxxx1, xxxx2 serveriu yra isjungti/uzblokuoti. Is networko puses mes galima pagelbet su firewallu taisyklem (srautai tarp skirtingu environmentu, potinkliu), bet vidiniu serveriu portu nustatymu nepriziurim. Jei reikalinga turet connectiona abi sitos vietos turi buti atidarytos (serveris klauso x portu ) + turi buti taisykle firewalle, jei kazkas is 'tolimo' tinklo nori connectintis i serveri tuo x portu.

skirtingi rulsetai ir praleidimai galioja skirtingiems environmentams ir enclave'ams, del to prode ir appvery esantys serveriai nebutinai tures tas pacias taisykles.

9 Logs

journalctl -u jupyterhub -f

journalctl -u rstudio-server.service -f

logs to file

ansible-playbook viya-ark-master/playbooks/viya-mmsu/viya-services-status.yml -vvv 2>&1 | tee -a viya_services_status_logs_20230727.txt
ansible-playbook viya-ark-master/playbooks/viya-mmsu/viya-services-stop.yml -e "enable_stray_cleanup=true" -vvv 2>&1 | tee -a viya_services_stop_logs_20230727.txt
ansible-playbook viya-ark-master/playbooks/viya-mmsu/viya-services-start.yml -vvv 2>&1 | tee -a viya_services_start_logs_20230727.txt

# sas installation logs to file

ansible-playbook site.yml -vvv 2>&1 | tee -a install_20230809_1500.txt

10 Yum

rpm official repo(yum, RHEL) commands - https://www.rpmfind.net/linux/RPM/ByName.html

# Use yum whatprovides to see what package provides a file.
yum whatprovides */ldapsearch
# Can see that ldapsearch is part of =openldap-clients=
# before installing we can check the content:
repoquery -lq openldap-clients
# all is good? install
yum install openldap-clients

yum history
yum list installed | grep devel
yum remove <reponame>
yum erase <reponame>
yum repolist all
yum repolist enabled
yum search 'udunit*' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8
yum search 'R-*' --disablerepo='*' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8
yum info 'R-devel' --disablerepo='*' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8
sudo yum install 'udunits2-devel' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8 --enablerepo=codeready-builder-for-rhel-8-x86_64-rpms
# list all available red hat repos
yum repolist all
# search for package in all repos + EPEL8 repo
yum search 'R-*' --disablerepo='*' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8
# search EPEL8 repo
yum info 'R-devel' --disablerepo='*' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8
# install from EPEL8 repo
sudo yum install 'R-devel' --enablerepo=hello_Extra_Packages_for_Enterprise_Linux_epel8 --enablerepo=codeready-builder-for-rhel-8-x86_64-rpms

11 Cron

crontab -l                    # view your own user's crontab
crontab -e                      # edit your own user's crontab
crontab -r                      # remove your own user's crontab:

12 zip

tar -cvf archive_name.tar files_or_directories_to_archive
tar -cvf labas.tar .

13 NAS

# list all drives?
cat /etc/fstab
# get mount name
mount -l |grep sas-store

14 odbc

For setting up ODBC connection you need temp root user. Add temp root user - Model Validation Home - SEB Confluence (sebank.se)

After you have your file you need to create global variable to point to that file location for ODBC to pic up needed information. "export ODBCINI=/etc/odbc.ini" (only for the first time).

nc -vz dbname 1433
ping dbname


# You can list the data sources on your system with the following command:
odbcinst -q -s

# NOTE. You can determine the location of the configuration files on your
# system by entering the following command into a terminal:
odbcinst -j

# Check what ODBC drivers are installed for required DB in "/etc/odbcinst.ini"
odbcinst -q -d


tsql -S dbname -U username
select name from sysdatabases
quit

15 Tmux commands

scroll in tmux - ctrl+b+[ then emacs bindings up and down

tmux - start new

tmux new -s myname - start new with session name

tmux a # - (or at, or attach) attach

tmux a -t myname - attach to named

tmux kill-session -t myname - kill session

In tmux, hit the prefix ctrl+b and then:

15.1 [A] List all shortcuts

to see all the shortcuts keys in tmux simply use the bind-key ? in my case that would be CTRL-B ?

15.2 Sessions

:new<CR>  new session
s  list sessions
$  name session
$  kill session

15.3 Windows (tabs)

c  create window
w  list windows
n  next window
p  previous window
f  find window
,  name window
&  kill window

15.4 Panes (splits)

%  vertical split
"  horizontal split
o  swap panes
q  show pane numbers
x  kill pane
; alternate between panes
+  break pane into window (e.g. to select text by mouse to copy)
-  restore pane from window
space - toggle between layouts
<prefix> q (Show pane numbers, when the numbers show up type the key to goto that pane)
<prefix> { (Move the current pane left)
<prefix> } (Move the current pane right)
<prefix> z toggle pane zoom

15.5 Sync Panes

You can do this by switching to the appropriate window, typing your Tmux prefix (commonly Ctrl-B or Ctrl-A) and then a colon to bring up a Tmux command line, and typing:

:setw synchronize-panes

You can optionally add on or off to specify which state you want; otherwise the option is simply toggled. This option is specific to one window, so it won’t change the way your other sessions or windows operate. When you’re done, toggle it off again by repeating the command. tip source

15.6 Resizing Panes

You can also resize panes if you don’t like the layout defaults. I personally rarely need to do this, though it’s handy to know how. Here is the basic syntax to resize panes:

PREFIX : resize-pane -D (Resizes the current pane down)
PREFIX : resize-pane -U (Resizes the current pane upward)
PREFIX : resize-pane -L (Resizes the current pane left)
PREFIX : resize-pane -R (Resizes the current pane right)
PREFIX : resize-pane -D 20 (Resizes the current pane down by 20 cells)
PREFIX : resize-pane -U 20 (Resizes the current pane upward by 20 cells)
PREFIX : resize-pane -L 20 (Resizes the current pane left by 20 cells)
PREFIX : resize-pane -R 20 (Resizes the current pane right by 20 cells)
PREFIX : resize-pane -t 2 -L 20 (Resizes the pane with the id of 2 left by 20 cells)

15.7 Copy mode:

Pressing PREFIX [ places us in Copy mode. We can then use our movement keys to move our cursor around the screen. By default, the arrow keys work. we set our configuration file to use Vim keys for moving between windows and resizing panes so we wouldn’t have to take our hands off the home row. tmux has a vi mode for working with the buffer as well. To enable it, add this line to .tmux.conf:

setw -g mode-keys vi

With this option set, we can use h, j, k, and l to move around our buffer.

To get out of Copy mode, we just press the ENTER key. Moving around one character at a time isn’t very efficient. Since we enabled vi mode, we can also use some other visible shortcuts to move around the buffer.

For example, we can use "w" to jump to the next word and "b" to jump back one word. And we can use "f", followed by any character, to jump to that character on the same line, and "F" to jump backwards on the line.

Function                vi             emacs
Back to indentation     ^              M-m
Clear selection         Escape         C-g
Copy selection          Enter          M-w
Cursor down             j              Down
Cursor left             h              Left
Cursor right            l              Right
Cursor to bottom line   L
Cursor to middle line   M              M-r
Cursor to top line      H              M-R
Cursor up               k              Up
Delete entire line      d              C-u
Delete to end of line   D              C-k
End of line             $              C-e
Goto line               :              g
Half page down          C-d            M-Down
Half page up            C-u            M-Up
Next page               C-f            Page down
Next word               w              M-f
Paste buffer            p              C-y
Previous page           C-b            Page up
Previous word           b              M-b
Quit mode               q              Escape
Scroll down             C-Down or J    C-Down
Scroll up               C-Up or K      C-Up
Search again            n              n
Search backward         ?              C-r
Search forward          /              C-s
Start of line           0              C-a
Start selection         Space          C-Space
Transpose chars                        C-t

15.8 Misc

d  detach
t  big clock
?  list shortcuts
:  prompt

15.9 Configurations Options:

# Mouse support - set to on if you want to use the mouse
* setw -g mode-mouse off
* set -g mouse-select-pane off
* set -g mouse-resize-pane off
* set -g mouse-select-window off

# Set the default terminal mode to 256color mode
set -g default-terminal "screen-256color"

# enable activity alerts
setw -g monitor-activity on
set -g visual-activity on

# Center the window list
set -g status-justify centre

# Maximize and restore a pane
unbind Up bind Up new-window -d -n tmp \; swap-pane -s tmp.1 \; select-window -t tmp
unbind Down
bind Down last-window \; swap-pane -s tmp.1 \; kill-window -t tmp

16 vim

:set paste
:set nopaste

:set nonu
:set nu

# no highlighting:

:noh

17 ansible

ansible-galaxy collection list

# install default version
ansible-galaxy collection install community.general

# install specific version:
ansible-galaxy collection install geerlingguy.apache,v1.0.0

7.0.1

to delete a package - delete it's folder here:

for example I deleted community.general by doing rm -rf community_old/ in home/instaluser.ansible/collections/ansible_collections