オンライン学習・ビデオ会議システム BigBlueButton on Docker (導入メモ)

New BigBlueButton on Docker

FreeSwitch Reference

https://freeswitch.org/confluence/


Channel Variables Catalog


Access Control List (ACL)


Sofia SIP Stack


Modules


Configuring FreeSWITCH

FreeSwitch fs_cliコマンド

fs_cli can’t connect to FreeSWITCH server remotely

Docker内部からFreeSwitchのコマンドラインツール fs_cli を利用するため、event_socket.conf.xml で各種条件を設定します。

以下Socket Clientの設定で、FreeSwitchのコンテナ内で fs_cli のインターフェイスに入るためには、ホスト、ポート、パスワードオプションを付与します。

# fs_cli -H 10.7.7.1 -P 8021 -p ClueCon

event_socket.conf.xml

<configuration name="event_socket.conf" description="Socket Client">
  <settings>
    <param name="nat-map" value="false"/>
    <param name="listen-ip" value="$${local_ip_v4}"/>
    <param name="listen-port" value="8021"/>
    <param name="password" value="ClueCon"/>
    <param name="apply-inbound-acl" value="loopback.custom"/>
    <!--<param name="stop-on-bind-error" value="true"/>-->
  </settings>
</configuration>

vars.xml : fs_cli コマンドにより freeswitchターミナル に入り eval コマンドにより変数確認

  <!--
      The following variables are set dynamically - calculated if possible by freeswitch - and
      are available to the config as $${variable}.  You can see their calculated value via fs_cli
      by entering "eval $${variable}" on freeswitch console.

      hostname ---> host pc name
      local_ip_v4 ---> 10.7.7.1
      local_mask_v4 ---> 255.255.255.0
      local_ip_v6 ---> ::1
      switch_serial ---> -ERR no reply
      base_dir ---> /opt/freeswitch
      recordings_dir ---> /opt/freeswitch/var/lib/freeswitch/recordings
      sound_prefix ---> /opt/freeswitch/share/freeswitch/sounds/en/us/callie
      sounds_dir ---> -ERR no reply
      conf_dir ---> /opt/freeswitch/etc/freeswitch
      log_dir ---> /opt/freeswitch/var/log/freeswitch
      run_dir ---> /opt/freeswitch/var/run/freeswitch
      db_dir ---> /opt/freeswitch/var/lib/freeswitch/db
      mod_dir ---> /opt/freeswitch/lib/freeswitch/mod
      htdocs_dir ---> /opt/freeswitch/share/freeswitch/htdocs
      script_dir ---> /opt/freeswitch/share/freeswitch/scripts
      temp_dir ---> /tmp
      grammar_dir ---> /opt/freeswitch/share/freeswitch/grammar
   ***certs_dir ---> /opt/freeswitch/etc/freeswitch/tls
      storage_dir ---> /opt/freeswitch/var/lib/freeswitch/storage
      cache_dir ---> /opt/freeswitch/var/cache/freeswitch
      core_uuid
      zrtp_enabled ---> -ERR no reply
      nat_public_addr ---> -ERR no reply
      nat_private_addr ---> -ERR no reply
      nat_type ---> -ERR no reply
      external_ip_v4 ---> 192.168.x.x(host pc local IPv4)
      external_ip_v6 ---> xx:xx:xx:xx(host pc local IPv6)
      domain ---> www.example.com
      domain_name ---> www.example.com

  -->

TLS認証用ディレクトリは >eval $${certs_dir} により、 /opt/freeswitch/etc/freeswitch/tls

または、-xオプションを付加して以下コマンドで確認。

# fs_cli -x 'eval $${certs_dir}'
/opt/freeswitch/etc/freeswitch/tls

ソースからビルドした場合のデフォルトのバイナリディレクトリは、/usr/local/freeswitch/bin/

# ls /usr/local/freeswitch/bin/                
freeswitch  fs_cli  fs_encode  fs_ivrd	fs_tts	fsxs  gentls_cert  tone2wav

よって、fs_cli の起動は、

# /usr/local/freeswitch/bin/fs_cli

グローバル変数確認

freeswitch> global_getvar
hostname=
local_ip_v4=
local_mask_v4=255.255.255.0
local_ip_v6=
base_dir=/usr/local/freeswitch
recordings_dir=/usr/local/freeswitch/recordings
sounds_dir=/usr/local/freeswitch/sounds
conf_dir=/usr/local/freeswitch/conf
log_dir=/usr/local/freeswitch/log
run_dir=/usr/local/freeswitch/run
db_dir=/usr/local/freeswitch/db
mod_dir=/usr/local/freeswitch/mod
htdocs_dir=/usr/local/freeswitch/htdocs
script_dir=/usr/local/freeswitch/scripts
temp_dir=/tmp
grammar_dir=/usr/local/freeswitch/grammar
fonts_dir=/usr/local/freeswitch/fonts
images_dir=/usr/local/freeswitch/images
certs_dir=/usr/local/freeswitch/certs
storage_dir=/usr/local/freeswitch/storage
cache_dir=/usr/local/freeswitch/cache
data_dir=/usr/local/freeswitch
localstate_dir=/usr/local/freeswitch
default_password=
sound_prefix=/usr/local/freeswitch/sounds/en/us/callie
domain=
domain_name=
hold_music=local_stream://moh
use_profile=external
rtp_sdes_suites=AEAD_AES_256_GCM_8|AEAD_AES_128_GCM_8|AES_CM_256_HMAC_SHA1_80|AES_CM_192_HMAC_SHA1_80|AES_CM_128_HMAC_SHA1_80|AES_CM_256_HMAC_SHA1_32|AES_CM_192_HMAC_SHA1_32|AES_CM_128_HMAC_SHA1_32|AES_CM_128_NULL_AUTH
zrtp_secure_media=true
global_codec_prefs=OPUS,G722,PCMU,PCMA,H264,VP8
outbound_codec_prefs=OPUS,G722,PCMU,PCMA,H264,VP8
xmpp_client_profile=xmppc
xmpp_server_profile=xmpps
bind_server_ip=auto
external_rtp_ip=
external_sip_ip=
unroll_loops=true
outbound_caller_name=FreeSWITCH
outbound_caller_id=0000000000
call_debug=false
console_loglevel=info
default_areacode=918
default_country=US
presence_privacy=false
au-ring=%(400,200,383,417);%(400,2000,383,417)
be-ring=%(1000,3000,425)
ca-ring=%(2000,4000,440,480)
cn-ring=%(1000,4000,450)
cy-ring=%(1500,3000,425)
cz-ring=%(1000,4000,425)
de-ring=%(1000,4000,425)
dk-ring=%(1000,4000,425)
dz-ring=%(1500,3500,425)
eg-ring=%(2000,1000,475,375)
es-ring=%(1500,3000,425)
fi-ring=%(1000,4000,425)
fr-ring=%(1500,3500,440)
hk-ring=%(400,200,440,480);%(400,3000,440,480)
hu-ring=%(1250,3750,425)
il-ring=%(1000,3000,400)
in-ring=%(400,200,425,375);%(400,2000,425,375)
jp-ring=%(1000,2000,420,380)
ko-ring=%(1000,2000,440,480)
pk-ring=%(1000,2000,400)
pl-ring=%(1000,4000,425)
ro-ring=%(1850,4150,475,425)
rs-ring=%(1000,4000,425)
ru-ring=%(800,3200,425)
sa-ring=%(1200,4600,425)
tr-ring=%(2000,4000,450)
uk-ring=%(400,200,400,450);%(400,2000,400,450)
us-ring=%(2000,4000,440,480)
bong-ring=v=-7;%(100,0,941.0,1477.0);v=-7;>=2;+=.1;%(1400,0,350,440)
beep=%(1000,0,640)
sit=%(274,0,913.8);%(274,0,1370.6);%(380,0,1776.7)
df_us_ssn=(?!219099999|078051120)(?!666|000|9\d{2})\d{3}(?!00)\d{2}(?!0{4})\d{4}
df_luhn=?:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14}|3[47][0-9]{13}|3(?:0[0-5]|[68][0-9])[0-9]{11}|6(?:011|5[0-9]{2})[0-9]{12}|(?:2131|1800|35\d{3})\d{11}
default_provider=example.com
default_provider_username=
default_provider_password=
default_provider_from_domain=example.com
default_provider_register=false
default_provider_contact=5000
sip_tls_version=tlsv1,tlsv1.1,tlsv1.2
sip_tls_ciphers=ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH
internal_auth_calls=true
internal_sip_port=5060
internal_tls_port=5061
internal_ssl_enable=false
external_auth_calls=false
external_sip_port=5080
external_tls_port=5081
external_ssl_enable=false
rtp_video_max_bandwidth_in=3mb
rtp_video_max_bandwidth_out=3mb
suppress_cng=true
rtp_liberal_dtmf=true
video_mute_png=/usr/local/freeswitch/images/default-mute.png
video_no_avatar_png=/usr/local/freeswitch/images/default-avatar.png
AT_EPENT1=0 0 0 -1 -1 0 -1 0 -1 -1 0 -1
AT_EPENT2=1 1 1 -1 -1 1 -1 1 -1 -1 1 -1
AT_CPENT1=0 -1 -1 0 -1 0 0 0 -1 -1 0 -1
AT_CPENT2=1 -1 -1 1 -1 1 1 1 -1 -1 1 -1
AT_CMAJ1=0 -1 0 0 -1 0 -1 0 0 -1 0 -1
AT_CMAJ2=1 -1 1 1 -1 1 -1 1 1 -1 1 -1
AT_BBLUES=1 -1 1 -1 -1 1 -1 1 1 1 -1 -1
ATGPENT2=-1 1 -1 1 -1 1 -1 -1 1 -1 1 -1
core_uuid=

fs_cliの終了: ctrl+d または、

freeswitch> /exit

BigBlueButton Docker v2.4

v2.4から公式にBigBlueButtonを構成する各ブロック(サービス)が独立してイメージ化されています。

docker-composeファイルテンプレート

docker-compose.tmpl.yml

AWS,GCP等のVMインスタンスへのインストールの際は、下記を参照してポートを開放する必要があります。
https://docs.bigbluebutton.org/admin/configure-firewall.html#configure-your-firewall

  • TCP/IP port 22 (for SSH)
  • TCP/IP ports 80/443 (for HTTP/HTTPS)
  • UDP ports in the range 16384 - 32768 (for FreeSWITCH/HTML5 RTP streams)

ポートの割当て範囲

FreeSWITCH : 16384 - 24576
Kurento : 24,577 - 32,768

https://docs.bigbluebutton.org/admin/customize.html#change-udp-ports

Specifically, FreeSWITCH uses the range 16384 - 24576, which is defined in /opt/freeswitch/etc/freeswitch/autoload_configs/switch.conf.xml

    <!-- RTP port range -->
    <param name="rtp-start-port" value="16384"/>
    <param name="rtp-end-port" value="24576"/>

Kurento uses the range 24577 - 32768, which is defined in /etc/kurento/modules/kurento/BaseRtpEndpoint.conf.ini

    minPort=24577
    maxPort=32768

TURNサーバを配置する場合のポート

On the coturn server, you need to have the following ports (in addition port 22) available for BigBlueButton clients to connect (port 3478 and 443) and for coturn to connect to your BigBlueButton server (32769 - 65535).

Ports Protocol Description
3478 TCP/UDP coturn listening port
443 TCP/UDP TLS listening port
32769-65535 UDP relay ports range

Docker-Composeファイル環境変数設定ファイル.envファイル作成スクリプトの実行

DockerイメージによりCertbotスタンドアローンモードで起動。TLS認証を取得(coturnで必要)後Docker-Compose起動に必要な .env ファイル作成のため下記スクリプトを実行。

$ ./scripts/setup
Should greenlight be included? (y/n): y
Should an automatic HTTPS Proxy be included? (y/n): n
Should a coturn be included? (y/n): y
Coturn needs TLS to function properly.
   Since automatic HTTPS Proxy is disabled,
   you must provide a relative or absolute path
   to your certificates.
Please enter path to cert.pem: ../letsencrypt/live/xx.xx.xx/cert.pem
Please enter path to key.pem: ../letsencrypt/live/xx.xx.xx/privkey.pem
Please enter the domain name: f9e.duckdns.org
Should the recording feature be included?
   IMPORTANT: this is currently a big privacy issues, because it will 
   record everything which happens in the conference, even when the button
   suggests, that it does not.
   make sure that you always get people's consent, before they join a room!
   https://github.com/bigbluebutton/bigbluebutton/issues/9202
Choice (y/n): y
Should a Prometheus exporter be included? (y/n): n
Should old recordings be removed? (y/n): y
Please enter max age(days) for keeping recordings: 1
Is xx.xx.xx.xx your external IPv4 address? (y/n): y
Unable to find image 'jwilder/dockerize:latest' locally
latest: Pulling from jwilder/dockerize
88286f41530e: Pulling fs layer
3cabffebe5fe: Pulling fs layer
88286f41530e: Download complete
88286f41530e: Pull complete
3cabffebe5fe: Verifying Checksum
3cabffebe5fe: Download complete
3cabffebe5fe: Pull complete
Digest: sha256:b4da07ad265d2dd83fc91a73f56df54def78ba3db5c4fb74cbb9b7e79b7b3c58
Status: Downloaded newer image for jwilder/dockerize:latest
--------------------------------------------------
configuration file .env got successfully created!

you can look through it for further adjusments
  $ nano .env

make sure to recreate the docker-compose.yml after each change
  $ ./scripts/generate-compose

to start bigbluebutton run
  $ docker-compose up -d

Nginxの設定ファイル

NginxのSSL(TLS)設定は下記内容を参照のこと。

certbot --nginx

Using Let’s Encrypt

https://docs.bigbluebutton.org/2.2/install.html#using-lets-encrypt

Nginxリバースプロキシ設定ファイルテンプレート

reverse-proxy.conf.template

map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
}
map $remote_addr $endpoint_addr {
    "~:"    [::1];
    default    127.0.0.1;
}

server {
  listen 443 ssl http2 default_server;
  listen [::]:443 ssl http2 default_server;
  server_name ${NGINX_HOST};

  ssl_certificate /etc/letsencrypt/live/${NGINX_HOST}/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/${NGINX_HOST}/privkey.pem;
  
  ssl_session_cache shared:SSL:10m;
  ssl_session_timeout 10m;
  ssl_protocols TLSv1.2 TLSv1.3;
  ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
  ssl_prefer_server_ciphers on;
  # on the host machine, "$ sudo openssl dhparam -out ./letsencrypt/dhp-4096.pem 4096"
  ssl_dhparam /etc/letsencrypt/dhp-4096.pem;

  access_log  /var/log/nginx/bigbluebutton.access.log;
  error_log /var/log/nginx/bigbluebutton.error.log;

  location / {
    proxy_http_version 1.1;
    proxy_pass http://$endpoint_addr:48087;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_cache_bypass $http_upgrade;
  }
}

Coturnコンテナログ確認

$ docker logs bbb-docker_coturn_1 
CONFIGURATION ALERT: You specified --lt-cred-mech and --use-auth-secret in the same time.
Be aware that you could not mix the username/password and the shared secret based auth methods. 

lt-cred-mechuse-auth-secret に置き換えられたため、lt-cred-mechturnserver.conf でコメントアウトとします

KurentoとFreeSwitchのファイヤーウォール対応

:bangbang:

Configure BigBlueButton to work with your firewall

https://docs.bigbluebutton.org/admin/configure-firewall.html#Configure_Firewall_

Configure FreeSWITCH for using SSL

https://docs.bigbluebutton.org/2.2/install.html#configure-freeswitch-for-using-ssl

BBB各コンポーネントDockerネットワークIP一覧

:bangbang:

Network Configuration

Services as configured.

Service Network IP address Other Option
bbb-web bbb-net 10.7.7.2
html5-backend-{{$i}} bbb-net 10.7.7.{{add 100 $i}} Port {{ add 4000 $i }}
html5-frontend-{{$i}} bbb-net 10.7.7.{{add 200 $i}} Port {{ add 4100 $i }}
freeswitch network_mode: host
nginx network_mode: host extra_hosts:
- “host.docker.internal:10.7.7.1”
- “core:10.7.7.2”
- “etherpad:10.7.7.4”
- “webrtc-sfu:10.7.7.10”
- “html5:10.7.7.11”
etherpad bbb-net 10.7.7.4
redis bbb-net 10.7.7.5
mongodb bbb-net 10.7.7.6
kurento network-mode: host
webrtc-sfu bbb-net 10.7.7.10 ports: - “127.0.0.1:3008:3008”
extra_hosts:
- host.docker.internal:10.7.7.1
- kurento:10.7.7.1
fsesl-akka bbb-net 10.7.7.14
apps-akka bbb-net 10.7.7.15
libreoffice bbb-net 10.7.7.7
periodic bbb-net 10.7.7.12
recordings bbb-net 10.7.7.16
webhooks bbb-net 10.7.7.17
https_proxy bbb-net network_mode: host
coturn network_mode: host
greenlight ports: 10.7.7.1:5000:80
prometheus bbb-net 10.7.7.33
networks:  
  bbb-net:  
    ipam:  
      driver: default  
      config:  
        - subnet: "10.7.7.0/24"

以下スクリプトにより、デフォルトでは ポート5060(TCP,UDP) へのアクセスが制限されていますが、.env ファイルの SIP_IP_ALLOWLIST を指定することで 5060(UDP) へのアクセス制限を回避できます。

bbb-docker/mod/freeswitch/entrypoint.sh

#!/bin/bash

# remove all SIP (port 5060) iptable rules
iptables -S INPUT | grep "\-\-dport 5060 " | cut -d " " -f 2- | xargs -rL1 iptables -D

# block requests to 5060 (tcp/udp)
iptables -A INPUT -p tcp --dport 5060 -s 0.0.0.0/0 -j REJECT
iptables -A INPUT -p udp --dport 5060 -s 0.0.0.0/0 -j REJECT

# allow some IPs 
IFS=',' read -ra ADDR <<< "$SIP_IP_ALLOWLIST"
for IP in "${ADDR[@]}"; do
    # process "$i"
    echo "allow port 5060/udp for $IP"
    iptables -I INPUT  -p udp --dport 5060 -s $IP -j ACCEPT
done
...
...

Bandwidth Requirements


AWSやGCPのインスタンスBBBを配置する場合のデータコストの目安にして下さい。

What are the bandwidth requirements for running a BigBlueButton server

You’ll need good upstream and downstream bandwidth from the server. We recommend 1 Gbits/second bandwidth in both directions.

When sharing a webcam as a moderator, BigBlueButton lets you select 320x240, 640x480, or 1280x720. For bandwidth calculations, each resolution corresponds (roughly) to a .25 Mbits/sec, 0.40 Mbits/sec, and 0.60 Mbits/sec video stream respectively.

For example, if you have a room with 5 users, each sharing their webcam at 320x240, then you can calculate the bandwidth usage as follows:

  • Y = .25 Mbits/sec
  • W = amount of webcams that are streaming
  • U = amount of users that are watching

For calculations:

  • server incoming bandwidth: W*Y
  • server outgoing bandwidth: W*(U-1)*Y (minus one since a broadcaster does not have to subscribe to his own stream)

For example, with 5 users in a room with 5 webcams streaming, the bandwidth calculation is as follows:

  • in: 5*.25 = 1.25 Mbits/sec incoming bandwidth needed to the server, or 3600*1.25 = 4.5 GBits/hr
  • out: 5*(5-1)*.25 = 5 Mbits/sec outgoing bandwidth needed from the server, or 3600*5 = 18 Gbits/hr

If you’d have a typical classroom situation 1 presenter broadcasting their webcam to 30 remote students, the calculation is as follows:

  • in: 1*.25 = .25 Mbits/sec incoming, or 3600*.25 = 0.9 GBits/hr
  • out: 1*(30-1)*.25 = 7.25 Mbits/sec outgoing, or 3600*7.25 = 26.1 GBits/hr

Large “cafe-style chatroom”: 20 viewers, 8 people broadcasting with a webcam:

  • in: 8*.25 = 2 Mbits/sec incoming, or 3600*2 = 7.2 GBits/hr
  • out: 8*(20-1)*.25 = 38 Mbits/sec outgoing, or 3600* = 136.8 Gbits/hr

bbb-web TURN/STUN Server Setting

https://docs.bigbluebutton.org/admin/setup-turn-server.html#configure-bigbluebutton-to-use-your-turn-server

BBBTURN/STUNサーバの条件設定は、以下のXMLファイルによる。86400は秒数=24時間。

bbb-docker/mod/bbb-web/turn-stun-servers.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

    <bean id="stun1" class="org.bigbluebutton.web.services.turn.StunServer">
        <constructor-arg index="0" value="{{ .Env.STUN_SERVER }}"/>
    </bean>

    {{if .Env.TURN_SERVER }}
        <bean id="turn0" class="org.bigbluebutton.web.services.turn.TurnServer">
            <constructor-arg index="0" value="{{ .Env.TURN_SECRET }}"/>
            <constructor-arg index="1" value="{{ .Env.TURN_SERVER }}"/>
            <constructor-arg index="2" value="86400"/>
        </bean>
    {{end}}

    <bean id="stunTurnService" class="org.bigbluebutton.web.services.turn.StunTurnService">
        <property name="stunServers">
            <set>
                <ref bean="stun1" />
            </set>
        </property>
        <property name="turnServers">
            <set>
                {{if .Env.TURN_SERVER }}
                <ref bean="turn0" />
                {{end}}
            </set>
        </property>
        <property name="remoteIceCandidates">
            <set>
            </set>
        </property>
    </bean>
</beans>

Customize

https://docs.bigbluebutton.org/admin/customize.html#Customize_

FreeSwitchのウェブソケットへのTLS接続、AWSインスタンスでのnftablesによるポートフォワードとファイヤーウォール設定Nginxによるリバースプロキシーの配置など、教育機関等の大規模システムから自宅などの小規模システムまで設置環境により異なるネットワーク設定事項を上手く調整できれば、高機能な会議システムを手に入れることが出来ます。
メイン画面

オンラインクラスでYouTube画面共有

モバイルからのアクセス

モバイルからオンラインクラスへ参加

:warning: 注) ネットワークのレベルゲージが赤になっているのは、あえて低スループットにしているためです。ホストマシンの処理能力次第ですが、数十人程度の同時参加は出来るようです。