Mirabellette.eu

A blog about digital independence and autonomy

Privacy

Host Firefox booksmarks with syncserver

Written by Mirabellette / 01 may 2018 / 4 comments

Introduction

In order to be more and more independent about my digital ecosystem. I decided to manage my bookmarks by myself. I use the browser Mozilla Firefox and Mozilla allows you to manage your own synchronization server. Syncserver stores only bookmarks, it didn't manage your Firefox account or the authentication mechanism. This could be done in installing Firefox Accounts Server but it is not the purpose of this tutorial.

firefox_logo

It was pretty annoying to deploy it because there aren't a lot of information available and it requires to me to do some basic stuff by myself. It took me something like 10 or 15 hours to make this article. By the way, it works now and you can read this tutorial and I hope you will find it useful :)

The Github repository isn't very active, one release in 2017 and 2016, two release in 2015 and 2014. Just add the Github repository to your RSS agregator to get news about update. If we trust the past, it shouldn't be done very often.

Configure the Firefox Synchronization Server

  • This setup was made on Debian stretch
  • To build the application, you need to be able to access to internet or to the python repository in order to download all dependencies includes in requirements.txt
  • Dependencies

    adduser --system --shell /usr/sbin/nologin --no-create-home firefox
    apt-get install python-dev git-core python-virtualenv g++ sqlite
    cd /opt
    sudo -u firefox git clone https://github.com/mozilla-services/syncserver

    Basic configuration

    The server is configured using an .ini file to specify various runtime settings. The file “syncserver.ini” is this file for the application. There is some setting that you must specify before building the application. Feel free to adjust the [server:main] part to your configuration. You can find the final syncserver.ini file here (some adjustment still required).

    The parameter public_url. You should modify it in order to match the interface where syncserver will be accessed by. Even if you run it inside a container or a virtual machine, you have to setup the public url.

    public_url = https://example.com

    The parameter sqluri. I choose to use a Sqlite database to store bookmarks because it is easy to backup. Feel free to use the one you want and modify the syncserver.ini. If you don't specify a Sql database, your bookmarks will be store in RAM and be reset each time you restart the server.Replace sqluri = sqlite:////tmp/syncserver.db by :

    sqluri = sqlite:////opt/syncserver/syncserver_data.db
    *//// means absolute path

    The parameter secret. It is better to generate a secret key for signing authentication tokens. If you don't, the server will generate it each time it start. That could mean a weak key if the random generator seed isn't good enough. Uncomment the parameter and set the value with the result of the next command:

    head -c 20 /dev/urandom | sha1sum

    The parameter allowed_issuers. If you are using the account system offered by Mozilla Firefox, you may want to restrict access to just that domain like so:

    allowed_issuers = api.accounts.firefox.com

    The parameter force_wsgi_environ. I setup the server behind an Apache2 reverse proxy. I make some try with false but it didn't work. I even open an issue in the official Github repository. The only to make it works was to set the force_wsgi_environ to true.

    force_wsgi_environ = true

    Build

    Don't skip the configuration step or your syncserver will not work as expected. As you build the application, you should configure syncserver.ini BEFORE build the application. If you don't, the modifications did to syncserver.ini will not be read.

    cd /opt/syncserver
    sudo -u firefox make build
    sudo -u firefox make test

    After that, if you run sudo -u firefox make serve, you should be able to see some lines about syncserver listening. It could tell you if something go wrong.

    Update

    After building the application, you could now see two new folder : syncserver.egg-info and local. You should delete them to be able to build the server again, for example for an update.

    rm -r syncserver.egg-info
    rm -r local

    Apache2 virtualhost

    I create a classic reverse proxy Apache2 virtual host. It just redirects flux to the virtual machine interface. You can find the script here.

    Configure your browser

    The procedure varies a little between desktop and mobile Firefox, and may not work on older versions of the browser. I will only describe the process for desktop version of firefox. Feel free to find more informations here

    • Enter “about:config” in the URL bar picture.

      about_config

      You should display this warranty screen, confirm your choice to continue.

      warranty

    • Made a research for “identity.sync.tokenserver.uri” as name. Double click on the line and replace the string by your public URL.

      tokenserver_uri

      The syntax should be like this https://example.com/token/1.0/sync/1.5.The current version is 1.7 but the endpoint didn't change ... * the original one is the one display in the previous picture https://token.services.mozilla.com/1.0/sync/1.5

    • Restart Firefox for the change to take effect.

    Note that this must be set prior to loading the sign-up or sign-in page in order to take effect, and its effects are reset on sign-out.

    Hardening and clean up

    Lock the instance for your own usage

    As you can see, you now use your own server to store your bookmarks. To avoid someone else could do that, you have to set the parameter allow_new_users to false in syncserver.ini and build the application again.

    vim syncserver.ini
    rm -r syncserver.egg-info
    rm -r local
    sudo -u firefox make build

    Systemd script

    Astonishingly, there is no Systemd script provides by the official tutorial. You could find the one I created here. You have to put it in /etc/systemd/system/ and execute systemctl daemon-reload then systemctl enable syncserver.service. It will start syncserver at each boot.

    Cleanup

    If you install make and g++ just for building this application, feel free to remove them.

    apt purge make g++

    Of course, setup the firewall in the correct way.

    Sources

    Social media

    If you find this article interesting, feel free to subscribe to my RSS flux and to follow me on Mastodon. Don't hesitate to share it if you think he could interest someone else.

    Fix openvpn mbuf packet dropped

    Written by Mirabellette / 11 april 2018 / no comments

    Hello everyone

    I hope you are going great and everything was fine for you.

    Use case

    I have an openvpn daemon running with TCP on 443 port on a Debian system. I got thousands of error messages since months about MBUF packet dropped. The message was

    openvpn MBUF: mbuf packet dropped

    It occurs only if you use openvpn over TCP. I know it is considered to be a bad practice because it unneeded traffic but you have to make choice.

    What I do

    After hours of research, I was able to fix it. I add this two lines to the server configuration file:

    tcp-queue-limit 4096
    bcast-buffers 4096

    Now, you have to restart openvpn with this command : systemctl restart openvpn or service openvpn restart if systemd isn't installed on your system.
    You should not see this message in log anymore \o/ and get a little bit more stable connection thanks to the undropped packet.

    Sources

    Social media

    If you find this article interesting, feel free to subscribe to my RSS flux and to follow me on Mastodon. Don't hesitate to share it if you think he could interest someone else.

    Advertising domain name blocking with Unbound

    Written by Mirabellette / 06 march 2018 / 6 comments

    I realized that Shaft made his script available here ... It is more powerful but also longer than this one because it makes some verification. To be honest, I think it is also better in some way. Feel free to combine them to make your own.

    Hello everyone,

    Today I want to talk to you about advertising in Internet and how to block a part of it with a domain name resolver like Unbound.

    You must be aware that there are thousands of way to track user's activities on internet. A good protection against this kind of things is to directly block the resolution of the domain which is trying to gather information about you. It is, of course, not perfect but it is a first good step to begin to reduce tracking about your online activity.

    Sometime I read journalduhacker.net, it is a website which gathering "good" article from French open source community. I found a very interesting article from Shaft about blocking a list of domain name with unbound. It is a very nice article which present how do it. It mention a very good trick to reduce the size of the ads list and the ram load of unbound. Thanks to him for his sharing. I just got a warning message with unbound, I don't know why but it works. I will investigate in it later and will of course tell you how to fix it. The warning message is like that:

    [1520173472] unbound[1259:0] warning: duplicate local-zone

    Unfortunately, I didn't find a script to modify ads list file from the source directly. They are commonly wrote like a host file. That's why I decided to made it by myself and to share it. I delete comments and other information in the original source file in a very strictly way. I do it in order to avoid any problem with Unbound. Some domain name could be deleted from the source list but with ~97400 domain name in it, I think the script I made works well enough.

    Most of ads list in the script are from Shaft article. I add this one too which is well reputed.
    Thanks to Sabre comment, I discovered that StevenBlack already provide an unique host list which contains AdAway, yoyo.org and MVPS hosts list. You can access to his list here. It is the one which is now in the script.

    vim /etc/unbound/unbound.conf.d/generate_domains_list_ban.sh

    # list of ads domain names
    array=( https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-gambling-porn-social/hosts )

    for i in "${array[@]}"
    do
      wget $i -O w
      grep -v " #\|<td>\|<p>\|<meta>\|<link>\|<title>\|href\|title=\|=\|<" w > adsList.txt
      rm w
      dos2unix adsList.txt

      # remove host syntax and clean file
      sed -i 's/0.0.0.0//g' adsList.txt
      sed -i 's/127.0.0.1//g' adsList.txt
      sed -i 's/localhost//g' adsList.txt
      sed -i 's/.localdomain//g' adsList.txt

      # remove commentary after domain name
      sed -i 's/#.*//' adsList.txt

      # remove tabulation character and carriage return
      sed -i "s/\t//g" adsList.txt
      sed -i "s/\r//g" adsList.txt

      # remove useless space
      sed -i 's/ //g' adsList.txt

      # remove empty lines
      sed -i '/^\s*$/d' adsList.txt

      # add prefix and suffix for unbound
      sed -i "s/.*/local-zone: \"&\" static/" adsList.txt

      cat adsList.txt >> adsListFinal.txt
    done

    # order list by name, it didn't cost a lot and could maybe increase unbound performance
    sort adsListFinal.txt -o adsListFinal.txt

    # remove duplicate ads domain in order to avoid warning with Unbound
    uniq adsListFinal.txt > adslist.txt

    # remove tempory files
    rm adsListFinal.txt adsList.txt

    service unbound restart

    You now have to tell to Unbound to load the advertise domain list. Add this line to /etc/unbound/unbound.conf and under the parameter server:

    # include: /YOUR_ADS_LIST_PATH
    include: /etc/unbound/unbound.conf.d/adslist.txt

    At the end of the process, I got a file of 4.1M with ~97400 domain names in it. Contrary to what we could think, It isn't slow. We just have to create a crontab job to be sure the list is oftenly updated. I think to update it each week is a good schedule.

    # 5 2 * * Sun /YOUR_GENERATE_ADS_LIST_SCRIPT_PATH
    5 2 * * Sun /etc/unbound/unbound.conf.d/generate_domains_list_ban.sh .sh

    It took me hours to make the script and this article. I hope you will find it useful and interesting. Don't hesitate to comment it and share it.
    Thank you for reading.

    sources

    Social media

    If you find this article interesting, feel free to subscribe to my RSS flux and to follow me on Mastodon. Don't hesitate to share it if you think he could interest someone else.

    Classified in : Privacy / Tags : none

    Specific OpenVPN error on Windows 10 "endpoints must exist within the same 255.255.255.252"

    Written by Mirabellette / 15 may 2017 / no comments

    Hello everyone,

      system
    • debian jessie stable 8.7
    • windows 10 creators update
    • openvpn 2.3.4.5 on the server
    • openvpn 2.4.2 on the client

    I recently got a new laptop with windows 10. Obviously, I installed OpenVPN with the correct configuration. When I try to connect it i got an error's message.

    if The local and remote VPN endpoints must exist within the same 255.255.255.252

    After some hours to find a solution, I finally find it and it is trivial. You just have to

    open your open vpn configuration
    add "topology subnet"
    restart openvpn

    vim /etc/openvpn/server.conf
    add "topology subnet"
    service openvpn restart

    * on the client

    service openvpn restart

    Now, the openvpn client must be able to connect to the server without error.

    I hope this article will help you to solve this kind of problem.