videobridge installation

Adding videobridges

Follow Jitsi Load balancing guide and the video https://youtu.be/LyGV4uW8km8

This has been deployed on several machines, see #42 (closed) . Bare metal preferred.

Prepare server

rename server to bridgeN in /etc/hosts and /etc/hostname and reboot

apply ansible playbook hardening.yml (in /etc/ansible). May require: ~~python apt-get install python-apt or~~ ln --symbolic /usr/bin/python3 /usr/bin/python

Change /ansible/hosts and test first with ansible all -m ping from your laptop. Then run with ansible-playbook playbook.yml

After that check SSH port an do a sshd -t after running the script so you do not lock yourself out! You better add missing keys:

/usr/bin/ssh-keygen -A or do each manually:

ssh-keygen -b 1024 -t rsa -f /etc/ssh/ssh_host_key
ssh-keygen -b 1024 -t rsa -f /etc/ssh/ssh_host_rsa_key 
ssh-keygen -b 1024 -t dsa -f /etc/ssh/ssh_host_dsa_key

Add host name and repo as in this guide first steps https://github.com/jitsi/jitsi-meet/blob/master/doc/quick-install.md#basic-jitsi-meet-install

Option 1: Jitsi stable release

Now works with default Java 11. ~~Install Java 8 from other repo #50 (comment 21586)~~

echo 'deb https://download.jitsi.org stable/' >> /etc/apt/sources.list.d/jitsi-stable.list
wget -qO -  https://download.jitsi.org/jitsi-key.gpg.key | apt-key add -
apt update
apt install jitsi-videobridge2

Option 2: Jitsi unstable repo

~~Has newer version of videobridge, which is considered stable and which should work fine with Java 11 (default in buster).~~

echo 'deb https://download.jitsi.org unstable/' >> /etc/apt/sources.list.d/jitsi-unstable.list
wget -qO -  https://download.jitsi.org/jitsi-key.gpg.key | apt-key add -
apt update
apt-get install jitsi-videobridge2

configure the new bridge as MUC

From here the video explains installing only videobridge on VM: https://youtu.be/LyGV4uW8km8?t=879 (component version)

enable MUC in /etc/jitsi/videobridge/sip-communicator.properties and colibri

# MUC variant
org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443
org.jitsi.videobridge.ENABLE_STATISTICS=true
org.jitsi.videobridge.STATISTICS_TRANSPORT=muc,colibri
org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=meet.fairkom.net
org.jitsi.videobridge.xmpp.user.shard.DOMAIN=auth.meet.fairkom.net
org.jitsi.videobridge.xmpp.user.shard.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.shard.PASSWORD=xxxyyy
org.jitsi.videobridge.xmpp.user.shard.MUC_JIDS=JvbBrewery@internal.auth.meet.fairkom.net
org.jitsi.videobridge.xmpp.user.shard.MUC_NICKNAME=4234327b-26a6-4c78-bc6e-cecd7588bc45
org.jitsi.videobridge.xmpp.user.shard.DISABLE_CERTIFICATE_VERIFICATION=true
org.jitsi.videobridge.ENABLE_REST_SHUTDOWN=true
org.jitsi.videobridge.shutdown.ALLOWED_SOURCE_REGEXP=127.0.0.1

not used parameters in jvb2 (was component until May 3 2020 on OVH server)

org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=turn.fairkom.net:443,meet-jit-si-turnrelay.jitsi.net:443
org.jitsi.videobridge.AUTHORIZED_SOURCE_REGEXP=focus@auth.fairmeeting.net/.*
org.jitsi.videobridge.ENABLE_STATISTICS=true
org.jitsi.videobridge.STATISTICS_TRANSPORT=muc,colibri,pubsub
org.jitsi.videobridge.PUBSUB_SERVICE=fairmeeting.net
org.jitsi.videobridge.PUBSUB_NODE=sharedStatsNode
org.jitsi.videobridge.STATISTICS_INTERVAL=5000
# org.ice4j.ice.harvest.ALLOWED_ADDRESSES=144.76.116.175
org.jitsi.videobridge.ENABLE_REST_SHUTDOWN=true

define in /etc/jitsi/videobridge/config

JVB_HOSTNAME=meet.fairkom.net
JVB_HOST=meet.fairkom.net
JVB_SECRET=secret
# extra options to pass to the JVB daemon
JVB_OPTS="--apis=rest,xmpp,colibri"

do some kernel performance tweaking

In /etc/systemd/system.conf add

# jitsi performance
DefaultLimitNOFILE=65000
DefaultLimitNPROC=65000
DefaultTasksMax=65000

increase the TCP and UDP receive buffer size
check default settings first with sysctl -a | grep mem

append in /etc/sysctl.conf

# increase Linux TCP buffer limits
net.core.rmem_max = 10485760
net.core.wmem_max = 10485760
net.core.netdev_max_backlog=100000

# increase Linux autotuning TCP buffer limits
# min, default, and max number of bytes to use
net.ipv4.tcp_mem = 382401 509869 764802
net.ipv4.tcp_rmem = 4096 131072	6291456
net.ipv4.tcp_wmem = 4096 16384 4194304

# increase Linux autotuning UDP buffer limits
net.ipv4.udp_mem = 764178 1018904 1528356

CoDel below not tested on all bridges, runs only on bridge2 as it comes with /etc/sysctl.d/30-peertube-tcp.conf

# In a video server, we are often sending files to a client
# which can't accept it as fast as our local network connection
# could produce packets. To prevent packet loss and buffer bloat,
# it's especially important to use a modern CoDel scheduler which
# knows how to delay outgoing packets to match slower client links.

net.core.default_qdisc = fq_codel
net.ipv4.tcp_congestion_control = bbr

load with sysctl -p

Assign more memory to videobridges

In /usr/share/jitsi-videobridge/lib/videobridge.rc

   VIDEOBRIDGE_MAX_MEMORY=8192m

or in /etc/jitsi/videobridge/config

   VIDEOBRIDGE_MAX_MEMORY=8192m

on fairmeeting main server

Let bridges communicate with MUC #55 (closed)

Previously we had to enter a bridge as component with its secret in /etc/prosody/conf.avail/fairmeeting.net.cfg.lua

Component "jvb-HOSTINGPARTNER-0.fairmeeting.net"
    component_secret = "secret"

and then do

service prosody restart service jitsi-videobridge restart

For statistics, check /var/log/jitsi/jicofo.log like https://youtu.be/LyGV4uW8km8?t=1269 for messages like Video stream count for:

allow colibri web socket

See #111 (comment 42337)

open port 9090 at all video bridges to be accessible from the run server

ufw allow from 157.90.234.18 proto tcp to any port 9090

add at /etc/jitsi/videobridge/jvb.conf

videobridge {
    http-servers {
        public {
            port = 9090
        }
    }
    websockets {
        server-id = "1.2.3.4"
        enabled = true
        domain = "run.fairmeeting.net:443"
        tls = true
    }
}

Use jvb's IP address for server-id - this will allow proxy in /etc/nginx/sites-available/run.fairmeeting.net.conf to proxy the client via websocket directly to the IP of the bridge so that the bridge can ask directly the web browser on the health.