quinta-feira, 5 de dezembro de 2019

Check your link speed to all servers in location

So I had same problems with my local xDSL provider.

To be able to test the speed of connection we can use the speedtest site or a bash script from a GitHub developer, but I created the following bash script which will use the speedtest script to test the speed on all the servers in my city and save it to a log file.

#!/bin/bash
#
# speedtest all my city servers
#
# Author: braselectron.com 05/DEZ/2019
#
#
# ref: 

# curl -s https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py | python -
#
# dependencies: 

#
#  1) wget https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py
#

#  2) ubuntu 18.04
#

#  2.1) GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)
#

#  3) http://ipinfo.io/ip   # for external IP fetch
#

#
targetcity="Miami"

# if target city name has spaces change it for grep \s like 
# this: "Salt\sLake\sCity"
gtarget=$( echo "$targetcity" | sed "s/ /\\\s/g )
#
servers=$( python speedtest.py --list | grep $gtarget )
# debug:
# echo -e "I found these servers:\n$servers"
servnum=$( echo -e "$servers" | cut -d ')' -f 1 )

# debug:
# echo -e "These are the server numbers:\n$servnum"
tot=$( echo -e "$servnum" | wc -l )
echo -e "\nRunning, please wait $tot servers to go:\n"

#
echo -e "Speedtest "$( wget -qO - http://ipinfo.io/ip )" "$( date +%F )"\n" > speedtest_$( date +%F ).log
#

i=1
#

for serv in $servnum
   do
     echo -e "$i of $tot - ($serv): "
     python speedtest.py --server $serv 2>&1 | grep ':' |\
     sed "s/$targetcity/$serv/g" |\
     sed "s/.*ERROR.*/Server is not responding/g" |\
     tee -a speedtest_$( date +%F ).log
     echo -e "" | tee -a speedtest_$( date +%F ).log
     sleep 1
     ((i++))
     #
   done
echo ""
Cheers!

sexta-feira, 22 de novembro de 2019

How to list, delete and include iptables rules.

First you should list all rules and index tem so you can pinpont your request correctly:

 sudo iptables -nL --line-numbers

Note: the -n means to not use DNS to resolve IP; the -L means to list rules; --line-numbers you can guess what it means.

Now you can choose a "chain", in this example: "INPUT" and use the "-D" to indicate delete and the final number is point to rule "5"

 sudo iptables -D INPUT 5

You can also add a rule, and bellow I will use chain INPUT for a known hacker from Kenya, Africa:

 sudo iptables -I INPUT 1 -s 197.248.0.0/16 -j DROP

That's it for now. 
Cheers!  and Shields up!

sábado, 26 de outubro de 2019

How to recover packages to local cache - Ubuntu

I had to sudo apt clean my local cache to uninstall a bad package and needed to recover my installed packages since I work often in offline places, so that I can recover from a broken system without Internet.

So be able to recover, or better, re-download packages to local cache that usually is located at: /var/cache/apt/archives  we have this:

1) get install package names:

dpkg -l | grep "^ii" | awk ' {print $2} ' > pkg_names.txt

2) get install package versions:

dpkg -l | grep "^ii" | awk ' {print $3} ' > pkg_version.txt

3) combine package names and versions

paste -d"=" pkg_names.txt pkg_version.txt > pkg_combine.txt

4) now re-download packages to cache

cat pkg_combine.txt | xargs sudo apt-get -y install --reinstall --download-only

REF:

dpkg -l | grep "^ii"| awk ' {print $2} ' | xargs sudo apt-get -y install --reinstall --download-only

dpkg -s firefox | grep Version | awk ' {print $2} '

sudo apt-get install package=version

Todo:

Timeout each line to avoid long loops (when source is dead or pkg not available at repository)


quarta-feira, 16 de outubro de 2019

List total hits of iptable drops

I have been busy making and studying defense strategies to better implement and monitor my firewall system.

So after you have decided to insert DROP rules in your iptables, it is a good practice to check the statistics of how efficient the additional "load" has been to your overall performance, that is, is it effective or should you leave it for fail2ban to control the hits ?

I made a bash script to summaries the hits on iptables rules to avoid all the stdout that obfuscates the  important information.

As usually sad, use at your own risk.  Make a backup plan before proceeding.

#!/bin/bash
#
# check how many hits on iptables drop rule
# by braselectron.com  OCT 16, 2019
#
# iptables formated output example needed:

# '0 0 DROP all -- any any 47.203.94.77 anywhere'
#

# so now clear and fix the spaces on the output
#
readarray iptls <<< "$(sudo iptables -vnL | grep DROP |\

 sed 's/  / /g' | sed 's/  / /g' | sed 's/  / /g' |\
 sed 's/^ //g')"
#
# debug point
# echo "iptls lenght is ${#iptls[@]}"
#
echo -e "Hits\tTarget Denied"
for element in "${iptls[@]}"
do

  #
  # debug point
  # echo "> $element"

  #
  verify=( $(echo "$element" | cut -d " " -f 1) )

  #
  # debug point
  # echo "verify = $verify"

  #
  if [ "$verify" -ne "0" ]; then
     hits="$verify"
     target=( $(echo "$element" | cut -d " " -f 8) )
     echo -e "$hits\t$target"
  fi
done

If all goes well you will get a output similar to this:

Hits    Target Denied
1       198.108.66.0/23
4       92.118.161.0/24
3       92.118.160.0/24
1       74.82.47.0/24
1       185.173.35.0/24
1       71.6.128.0/17
2       122.228.0.0/16
4       95.154.101.209
1131    139.59.13.150

But this is based on my active iptables rules.

Shields up captain!
Cheers!

How to fix id3tag of mp3 files

Preliminary notes:

This post is the sum of many hours of researching the web for solutions, reading for many hours: man, info and community pages. And a lot of testing.

This is a solution for Linux users (probably also good for MacOS too).  MS Windows user, sorry, you need to contact MS support for help.

Part of the following is cut/past form other sources I found on the web.  Thank you for the active community of Linux users like me.

As usually noted, the following is for sharing knowledge only, do it at your own risk!  Make a backup plan before proceeding.

Basic knowledge:

Libavformat (lavf) is a library for dealing with various media container formats.  Its main two purposes are demuxing - i.e. splitting a media file into component streams, and the reverse process of muxing - writing supplied data in a specified container format.

The MP3 muxer writes a raw MP3 stream with the following optional features:

An ID3v2 metadata header at the beginning (enabled by default). Versions 2.3 and 2.4 are supported, the id3v2_version private option controls which one is used (3 or 4).

Setting id3v2_version to "0" disables the ID3v2 header completely.

The muxer usually support writing attached pictures (APIC frames) to the ID3v2 header. The pictures are supplied to the muxer in form of a video stream with a single packet. There can be any number of those streams, each will correspond to a single APIC frame. The stream metadata tags title and comment map to APIC description and picture type respectively.

See http://id3.org/id3v2.4.0-frames for allowed picture types.

Note that the APIC frames must be written at the beginning, so the muxer will buffer the audio frames until it gets all the pictures. It is therefore advised to provide the pictures as soon as possible to avoid excessive buffering.  Also keep picture small (ie. less than 1% of total mp3 size is a good rule/target to follow).

A Xing/LAME frame right after the ID3v2 header (if present). It is enabled by default, but will be written only if the output is seekable. The write_xing private option can be used to disable it. The frame contains various information that may be useful to the decoder, like the audio duration or encoder delay.

A legacy ID3v1 tag at the end of the file (disabled by default). It may be enabled with the write_id3v1 private option, but as its capabilities are very limited, its usage is not recommended.  This is important for old mp3 players that only understand ID3v1 tags.


Examples:

On the next part you must change the input, out.mp3 and cover.png to your own fit.

1) Write an mp3 with an ID3v2.3 header and an ID3v1 footer:

ffmpeg -i input.mp3 -id3v2_version 3 -write_id3v1 1 out.mp3

2) To attach a picture to an mp3 file select both the audio and the picture stream with map:

ffmpeg -i input.mp3 -i cover.png -c copy -map 0 -map 1 \
-metadata:s:v title="Album cover" -metadata:s:v comment="Cover (Front)" out.mp3

3) Write a "clean" MP3 without any extra features:

ffmpeg -i input.wav -write_xing 0 -id3v2_version 0 out.mp3


Let's get work done:
So this is what I did to add a cover image to a mp3 that already had id3tags:

ffmpeg -i orignal.mp3 -i cover.jpg -map 0:0 -map 1:0 -c copy -id3v2_version 3 -metadata:s:v comment="Cover (Front)" final.mp3


Other thoughts:

I did use a jpeg image for the cover and ffmpeg accepted it as well as vlc, mpv and other players I tested did work fine.

So if you need to include the same cover on many files (ie an album) in a folder, you can do this:

1) "cd" to the target folder
2) "mkdir new" in the target folder
3) rename the cover image file to cover.<ext> where <ext> could be png or jpg depends on your choice and particular case, or convert it to jpg.
3) then copy and past the following script (and hit the [ENTER] key):

for file in *.mp3; do ffmpeg -i "${file}" -i cover.jpg \
 -map 0:0 -map 1:0 -c copy -id3v2_version 3 -metadata:s:v \
 comment='Cover (Front)' ./new/"${file}"; done

After it runs you will have a sub-folder with the new mp3 files all with the embedded cover image you inserted with the script.

Have fun!  Cheers!

quarta-feira, 25 de setembro de 2019

Check your servers are UP - Linux BASH

Recently I was trying to check if one of my servers had a glitch, so after some research and coding I came up with this solution for Linux (ie. in my case for Raspbian Jessie):

1) Get the following code and save it to your target server.

#!/bin/bash
#
# Test connectivity with ping

# filename: test_alive.sh
#
# braselectron.com - September 25, 2019
#
# get IP from command line argument
#
ip=$1
#
# check IP address format code
# Mitch Frazier - Linux Journal - June 26, 2008
#
valid_ip () {
    local  ip=$1
    local  stat=1
    if [[ $ip =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
        OIFS=$IFS
        IFS='.'
        ip=($ip)
        IFS=$OIFS
        [[ ${ip[0]} -le 255 && ${ip[1]} -le 255 && ${ip[2]} -le 255 && ${ip[3]} -le 255 ]]
        stat=$?
    fi
    return $stat
}
#
# check syntax
#
if [ $# -eq 0 ]
   then echo "syntax: test_live.sh <ip address>"
   exit 1
fi
#
if ! valid_ip $ip
   then echo "IP is invalid"
   exit 1
fi
#
# ping but don't wait
# log if ping fails
#
while true
   do ping -w 1 -c 1 $ip |\
      grep received |\
      cut -d" " -f 4 |\
      if [ "$(cat -)" != "1" ]
         then echo "Ping failed $(date)"
      fi
   done
 

 2) Choose another host on your network to ping that is up (for sure) and is trusted by you.  For example: 192.168.0.1 (usually the router on your network - default factory value).

3) Now using "nohup" to setup your test on the target host, at the command terminal, do this:

nohup  /bin/bash test_alive.sh 192.168.0.1 &
# remember to rename the 192.168.0.1 with your host IP (step 2)

4) Now you can logoff and from time to time check the nohup.out file to see if the ping failed.

5) When you are satisfied with your tests just kill the process on the target host, doing this:

5.1) first find your process with:  ps -ef | grep test_alive

This will give you a output similar to this:

pi        1257  4688  1 17:59 pts/1    00:00:00 grep --color=auto test_alive
pi       22213     1  9 15:08 ?        00:16:24 /bin/bash test_alive.sh 192.168.0.1


5.2) get the process id and kill it with:  kill 22213  # caution! use your id number

5.3) and remove the  nohup.out file with: rm nohup.out

NOTE:  if the nohup.out file is empty, means no erros, your server is working and connectivity is working.  But if the target host is freezing or other problems that make the script stop, this may mislead your conclusion.

Cheers!