Mirabellette.eu

A blog about digital independence and autonomy

Fix openvpn mbuf packet dropped

Written by Mirabellette / 11 april 2018 / no comments

Hello everyone

I hope you are going great and everything was fine for you.

Use case

I have an openvpn daemon running with TCP on 443 port on a Debian system. I got thousands of error messages since months about MBUF packet dropped. The message was

openvpn MBUF: mbuf packet dropped

It occurs only if you use openvpn over TCP. I know it is considered to be a bad practice because it unneeded traffic but you have to make choice.

What I do

After hours of research, I was able to fix it. I add this two lines to the server configuration file:

tcp-queue-limit 4096
bcast-buffers 4096

Now, you have to restart openvpn with this command : systemctl restart openvpn or service openvpn restart if systemd isn't installed on your system.
You should not see this message in log anymore \o/ and get a little bit more stable connection thanks to the undropped packet.

Sources

Social media

I hope you find find this article useful, don't hesitate to comment it and share it.

Classified in : OpenVPN / Tags : none

Check if it is possible to establish a SSH connection with Bash

Written by Mirabellette / 20 march 2018 / no comments

Hello everyone

I am continuing to write an article each month. I add some little title and subtitle to improve the ease of reading. Today, I want to share to you a little trick I use to check if it is possible to establish a SSH connection with a remote host.

Introduction

Use case

  • You want to transfer some files with SSH protocol. You want to be sure it is possible to establish a connection and be notified if it is not possible.
  • You want to check periodically if it is possible to connect to a remote host with SSH.

What I do

I created a bash script to open a connection with the remote host in SSH and check if it works well. If it doesn't, I send an email to a specific address.

The main part

Scripts

Both scripts are available in Github following MIT LICENCE. You can find them here.

Script to check ssh connection

script

Crontab

crontab

Requirement and advice

  • You need to have a bash prompt after you connected to ssh.
  • You need to have a mail transfer agent properly configured.
  • You need to check when you established a SSH connection that you don't receive any warning message from SSH. In this case, the status variable will got another value than "ok" and the script will considered you aren't be able to establish a connection.

Sources

Conclusion

I know that it must be a better way to test that but it fit well for my use cases.

Social media

I hope you find find this article useful, don't hesitate to comment it and share it.

Classified in : Tricks / Tags : none

Advertising domain name blocking with Unbound

Written by Mirabellette / 06 march 2018 / 6 comments

I realized that Shaft made his script available here ... It is more powerful but also longer than this one because it makes some verification. To be honest, I think it is also better in some way. Feel free to combine them to make your own.

Hello everyone,

Today I want to talk to you about advertising in Internet and how to block a part of it with a domain name resolver like Unbound.

You must be aware that there are thousands of way to track user's activities on internet. A good protection against this kind of things is to directly block the resolution of the domain which is trying to gather information about you. It is, of course, not perfect but it is a first good step to begin to reduce tracking about your online activity.

Sometime I read journalduhacker.net, it is a website which gathering "good" article from French open source community. I found a very interesting article from Shaft about blocking a list of domain name with unbound. It is a very nice article which present how do it. It mention a very good trick to reduce the size of the ads list and the ram load of unbound. Thanks to him for his sharing. I just got a warning message with unbound, I don't know why but it works. I will investigate in it later and will of course tell you how to fix it. The warning message is like that:

[1520173472] unbound[1259:0] warning: duplicate local-zone

Unfortunately, I didn't find a script to modify ads list file from the source directly. They are commonly wrote like a host file. That's why I decided to made it by myself and to share it. I delete comments and other information in the original source file in a very strictly way. I do it in order to avoid any problem with Unbound. Some domain name could be deleted from the source list but with ~97400 domain name in it, I think the script I made works well enough.

Most of ads list in the script are from Shaft article. I add this one too which is well reputed.
Thanks to Sabre comment, I discovered that StevenBlack already provide an unique host list which contains AdAway, yoyo.org and MVPS hosts list. You can access to his list here. It is the one which is now in the script.

vim /etc/unbound/unbound.conf.d/generate_domains_list_ban.sh

# list of ads domain names
array=( https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-gambling-porn-social/hosts )

for i in "${array[@]}"
do
  wget $i -O w
  grep -v " #\|<td>\|<p>\|<meta>\|<link>\|<title>\|href\|title=\|=\|<" w > adsList.txt
  rm w
  dos2unix adsList.txt

  # remove host syntax and clean file
  sed -i 's/0.0.0.0//g' adsList.txt
  sed -i 's/127.0.0.1//g' adsList.txt
  sed -i 's/localhost//g' adsList.txt
  sed -i 's/.localdomain//g' adsList.txt

  # remove commentary after domain name
  sed -i 's/#.*//' adsList.txt

  # remove tabulation character and carriage return
  sed -i "s/\t//g" adsList.txt
  sed -i "s/\r//g" adsList.txt

  # remove useless space
  sed -i 's/ //g' adsList.txt

  # remove empty lines
  sed -i '/^\s*$/d' adsList.txt

  # add prefix and suffix for unbound
  sed -i "s/.*/local-zone: \"&\" static/" adsList.txt

  cat adsList.txt >> adsListFinal.txt
done

# order list by name, it didn't cost a lot and could maybe increase unbound performance
sort adsListFinal.txt -o adsListFinal.txt

# remove duplicate ads domain in order to avoid warning with Unbound
uniq adsListFinal.txt > adslist.txt

# remove tempory files
rm adsListFinal.txt adsList.txt

service unbound restart

You now have to tell to Unbound to load the advertise domain list. Add this line to /etc/unbound/unbound.conf and under the parameter server:

# include: /YOUR_ADS_LIST_PATH
include: /etc/unbound/unbound.conf.d/adslist.txt

At the end of the process, I got a file of 4.1M with ~97400 domain names in it. Contrary to what we could think, It isn't slow. We just have to create a crontab job to be sure the list is oftenly updated. I think to update it each week is a good schedule.

# 5 2 * * Sun /YOUR_GENERATE_ADS_LIST_SCRIPT_PATH
5 2 * * Sun /etc/unbound/unbound.conf.d/generate_domains_list_ban.sh .sh

It took me hours to make the script and this article. I hope you will find it useful and interesting. Don't hesitate to comment it and share it.
Thank you for reading.

sources

Classified in : Privacy / Tags : none

Some news about the blog

Written by Mirabellette / 28 february 2018 / no comments

Hello everybody
I hope you are going fine. Today I would like to talk about the blog and his life. I don't know if you see it but I made some chances.
  • Installation of a new theme created by Dada. Thanks to him for his wonderful works. If you have some time and speak French, feel free to visit his blog.
  • Creation of a Mastodon account in laquadrature instance. La Quadrature du Net is a French non-profit association that defends the rights and freedom of citizens on the Internet. I will use the Mastodon account to communicate about news articles and share some interesting links I find on internet. I wrote a first "Pouet" some days ago.
  • Creation of a Media section, you can see it on the right above the category section.
  • After some time to think about it, I decided to create three new categories. The first one is about Digital independence, this category will welcome articles about self hosting or thought relative to that. The second new one is about Privacy and it will welcome article about privacy (Obvious isn't it? :p). The last one is about the blog himself, If I wanted to start a new project for example.
  • I changed category from published articles in order to be more logical and coherent..
  • I changed the subtitle of the blog by A blog about digital independence and autonomy..
  • I rewrite the services page with advice and warning about when to use services.
Concerning the future, I will continue to made tutorial about services you can host for yourself. I will also work in order to make the blog known a little bit more. I don't have a lot of visit now. I would like to increase this in order to help other people and maybe one day earn money with it. About the money part, I will publish a lot of things about how I see things. In all case, it is something very far from the current situation. ^^ I also want to create a community to share useful and interesting information with respect and tolerance. I am exhausted to read criticize and comments without any analyze or taking a moment to reflect. I want to bring something different, interesting things, quality and respect.

I hope you will like this changes. Feel free to use services I offered here, especially my Searx instance. It increases privacy for all of us if there are different user who use it. More information about services available are in the services static page.

Of course, don't hesitate to follow me on Mastodon.

Classified in : Blog / Tags : none

Installing and configuring Ethercalc in a LXC container

Written by Mirabellette / 09 february 2018 / no comments

Disclaimer
Installing and configuring a server is not something easy. It requires time, perseverance, money and knowledge. Don't forget that your server , Raspberry or I don't know stuff could be compromised and used, for example, against yourself or in a botnet network (like Mirai).

Since years, I used an instance of Ethercalc hosted by framasoft. EtherCalc is a web spreadsheet wh ich could be used by multiple user. It is quite powerful and you can, for example, manage most of your accountability with it. If you want to know more, yo u can test Ethercalc here.

If you have to give it back, feel free to do a little donation to Framasoft. They host a lot of very usefull services and works to empowered people. Even 10 (dollars, euros, something) could make a difference. You can also help in developing Ethercalc features in or helping to fix bugs.

In order to understand and control services I use, I decided to install an instance of Ethercalc. To give back to the community, I create this article in order to explain how I did.

System Configuration:
@host Debian Stretch (with apache as reverse proxy)
@installation_container Installation container (lxc)
@production_container Production container (lxc)

It is quite similar if you only have once environment.

Advices

  • Test this tutorial in a local network which is not directly connected to internet. I mean, protected by firewall.
  • Use a dedicated machine to do all of your test. A dedicated machine could be an unused computer, a virtual machine or a container. Be ready to reinstall your system, this could happened, especially when you begin with computer science.
  • I recommended to use another Linux container to build an application. In general, you have to avoid to install compilator and building application in a production machine. You must only have in production what it is required to work, nothing more! This reduce significantly the risk to be hacked.
  • In a container, I copy everything I need in /opt. It helps me to easily administrate container because I know everything I need to run the application is stored here.
  • All variable that you have to change are prefixed with $, you have to remove the $ too.

Let's get started!
First of all, before installing Ethercalc, it is recommended to use a Redis server to manage Ethercalc data.

Warning
All information stored by Redis are stored in clear. That means if someone is able to access to the dump.rdb file, he will be able to read all informations stored in your Ethercalc. I didn't find a solution to encrypt data from Redis and it doesn't seem to be developed yet.

# @installation_container

cd /opt
apt install xz-utils gcc make tar
wget http://download.redis.io/releases/redis-4.0.7.tar.gz
tar xvf redis-4.0.7.tar.gz
cd redis-4.0.7
cd deps
make hiredis jemalloc linenoise lua geohash-int
cd ..
make install

cd /opt
wget https://nodejs.org/dist/v8.9.4/node-v8.9.4-linux-x64.tar.xz
tar xvf node-v8.9.4-linux-x64.tar.xz
ln -s /opt/node-v8.9.4-linux-x64/bin/node /bin/node
/opt/node-v8.9.4-linux-x64/bin/npm install ethercalc
vim /opt/node-v8.9.4-linux-x64/lib/node_modules/ethercalc/bin/ethercalc
replace #!/bin/node by /opt/node-v8.9.4-linux-x64/bin/node/bin/node
# we don't need that node is available for all now, we will delete the symbolic link. rm /bin/node

mkdir /container_path/opt/redis-4.0.7 cp -r /opt/redis-4.0.7/src/redis-server /container_path/opt/redis-4.0.7/
cp -r /opt/node-v8.9.4-linux-x64/ /container_path/opt/node-v8.9.4-linux-x64/
# you can delete original source instead of just copying it

We are going to configure Redis in order to store data from Ethercalc where we want. As usual, I stored everything I need in /opt.

mkdir /container_path/opt/redis_data
wget http://download.redis.io/redis-stable/redis.conf -O /container_path/opt/redis.conf
# replace dir ./ by dir /opt/redis.conf

WARNING Redis doesn't implement server side encryption, that means that all your data are accessible from someone who can read the dump .rdb file.

Now, we will create two systemd script to start Ethercalc and Redis automatically each time the container start. We also configure iptables in order to avoid that Redis server is accessible from everywhere.

# @production_container

useradd redis
useradd nodejs
# we will now modify /etc/passwd in order to reduce user right to the strict minimum.
vim /etc/passwd
nodejs:x:1000:1000::/opt/node-v8.9.4-linux-x64/bin/node:/bin/false
redis:x:1001:1001::/opt/redis-4.0.7/redis-server:/bin/false

cd /opt/
chown redis:redis -R redis*
chown nodejs:nodejs -R node-v8.9.4-linux-x64/

vim /etc/systemd/system/redis.service
[Unit]
Description=Redis
After=network.target
[Service]
Type=simple
ExecStart=/opt/redis-4.0.7/redis-server /opt/redis.conf
RemainAfterExit=yes
User=nodejs
Group=nodejs

[Install]
WantedBy=multi-user.target
systemctl enable redis.service
vim /etc/systemd/system/ethercalc.service
[Unit]
Description=Ethercalc
After=network.target

[Service]
Type=simple
ExecStart=/opt/node-v8.9.4-linux-x64/lib/node_modules/ethercalc/bin/ethercalc --host $container_ip
RemainAfterExit=yes
User=nodejs
Group=nodejs

[Install]
WantedBy=multi-user.target

systemctl enable redis.service
# don't forget to change $container_ip by your own interface

In order to have iptables automatically loads when the container start, you have to install the package iptables-persistent. Of course, we will configure it in order to only have Ethercalc accessible from outside.

apt install iptables-persistent
vim /etc/iptables/rules.v4
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
COMMIT

*filter
:INPUT DROP [1:328]
-A INPUT -i lo -j ACCEPT
-A INPUT -s $container_ip -p tcp --dport 8000 -j ACCEPT
-A INPUT -s $apt_cache_or_lxc_network -m conntrack --ctstate ESTABLISHED -j ACCEPT
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT

Now we will configure the apache2 reverse proxy. I choose not to use HTTPS communication between $host and $production_container. The main reason is that both of them are hosted in the same computer. If $production_container was somewhere else on Internet, you MUST configure the proxy in order to have HTTPS communication between your container/virtual machine/whatever or all your traffic will be send in clear text on the network. I also added a web authentication in order to be the only one who can access to it. Feel free to change the $your_name by one of your choice. You have to change the appache2 configuration too. I will not explain how to deploy TLS on your web server, you can find a lot of tutorial in Internet.

# @host
# generate a password for $your_name
htpasswd -c /etc/apache2/password_ethercalc $your_name

vim /etc/apache2/site-available/ethercalc.conf
<VirtualHost *:443>
 ServerName www.example.com

 ErrorLog ${APACHE_LOG_DIR}/error.log
 CustomLog ${APACHE_LOG_DIR}/access.log combined


  AuthType Basic
  AuthName "Restricted Files"
  AuthBasicProvider file
  AuthUserFile "/etc/apache2/password_ethercalc"
  Require user $your_name


 SSLEngine on
 SSLCertificateFile /path/to/apache/crt
 SSLCertificateKeyFile /path/to/apache/key

 ProxyPass / "http://$container_ip:8000/"
 ProxyPassReverse / "http://$container_ip:8000/"

</VirtualHost>

ln -s /etc/apache2/site-available/ethercalc.conf /etc/apache2/site-enabled/ethercalc.conf
service apache2 reload

It took me hours to make this article. I hope you will find it useful and interesting Don't hesitate to comment, even if it is about mistake or something that could be improved.
Thank you for reading.

sources