聚合国内IT技术精华文章,分享IT技术精华,帮助IT从业人士成长

使用Docker安装多用户版的JupyterHub

2019-12-15 23:30 浏览: 3808840 次 我要评论(0 条) 字号:

宿主服务器使用的是Ubuntu 18.04,需要注意的是Docker目前不支持Ubuntu 19.10。如要在19.10中使用Docker需要在Docker源配置时设置Ubuntu 18.04的版本标识:bionic。

deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable

Docker的安装

Docker的安装流程非常的简单,按以下命令执行即可:

sudo apt remove docker docker-engine docker.io
sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add –
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt update
sudo apt install docker-ce
sudo usermod -aG docker $USER  #将当前用户加入到Docker组
sudo echo "DOCKER_OPTS="--registry-mirror=https://registry.docker-cn.com"" >> /etc/default/docker  #更改为国内源
sudo service docker restart

参考链接:

Docker中使用GPU

安装好了普通的Docker以后,如果想在容器内使用GPU会非常麻烦,好在Nvidia为了让大家能在容器中愉快使用GPU,基于Docker开发了Nvidia-Docker,使得在容器中深度学习框架调用GPU变得极为容易

参考链接:https://github.com/NVIDIA/nvidia-docker

多用户jupyterhub的安装

1、拉取相关镜像:

不用拉取latest版本,latest版本存在Bug,安装完成后不能正常运行。别问我怎么知道的,血与泪~

docker pull jupyterhub/jupyterhub:1.0.0
docker pull jupyterhub/singleuser:1.0.0

2、创建jupyterhub_network网络

docker network create --driver bridge jupyterhub_network

3、创建jupyterhub的volume

sudo mkdir -pv /data/jupyterhub
sudo chown -R root /data/jupyterhub
sudo chmod -R 777 /data/jupyterhub

4、创建jupyterhub_config.py文件并将其复制到volume

cp jupyterhub_config.py /data/jupyterhub/jupyterhub_config.py

文件内容:

# Configuration file for Jupyter Hub

c = get_config()

# spawn with Docker
c.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner'

# Spawn containers from this image
c.DockerSpawner.image = 'qw/jupyter_lab_singleuser:latest'

# JupyterHub requires a single-user instance of the Notebook server, so we
# default to using the `start-singleuser.sh` script included in the
# jupyter/docker-stacks *-notebook images as the Docker run command when
# spawning containers.  Optionally, you can override the Docker run command
# using the DOCKER_SPAWN_CMD environment variable.
c.DockerSpawner.extra_create_kwargs.update({ 'command': "start-singleuser.sh --SingleUserNotebookApp.default_url=/lab" })

# Connect containers to this Docker network
network_name = 'jupyterhub_network'
c.DockerSpawner.use_internal_ip = True
c.DockerSpawner.network_name = network_name
# Pass the network name as argument to spawned containers
c.DockerSpawner.extra_host_config = { 'network_mode': network_name }

# Explicitly set notebook directory because we'll be mounting a host volume to
# it.  Most jupyter/docker-stacks *-notebook images run the Notebook server as
# user `jovyan`, and set the notebook directory to `/home/jovyan/work`.
# We follow the same convention.
notebook_dir = '/home/jovyan'
# notebook_dir = '/home/jovyan/work'
c.DockerSpawner.notebook_dir = notebook_dir
# Mount the real user's Docker volume on the host to the notebook user's
# notebook directory in the container
c.DockerSpawner.volumes = { 'jupyterhub-user-{username}': notebook_dir,'jupyterhub-shared': {"bind": '/home/jovyan/shared', "mode": "rw"}}
# volume_driver is no longer a keyword argument to create_container()
# c.DockerSpawner.extra_create_kwargs.update({ 'volume_driver': 'local' })
# Remove containers once they are stopped
c.DockerSpawner.remove_containers = True
# For debugging arguments passed to spawned containers
c.DockerSpawner.debug = True

# The docker instances need access to the Hub, so the default loopback port doesn't work:
# from jupyter_client.localinterfaces import public_ips
# c.JupyterHub.hub_ip = public_ips()[0]
c.JupyterHub.hub_ip = 'jupyterhub'

# IP Configurations
c.JupyterHub.ip = '0.0.0.0'
c.JupyterHub.port = 8000

# OAuth with GitLab
import os
c.JupyterHub.authenticator_class = 'oauthenticator.gitlab.GitLabOAuthenticator'

os.environ['OAUTH_CALLBACK_URL'] = 'http://10.101.14.13:8000/hub/oauth_callback'
os.environ['GITLAB_CLIENT_ID'] = 'd89d76ef002100f217f4a7c1fc73011ca4d9eee7bb5ff8ce3e9532ba7721e29e'
os.environ['GITLAB_CLIENT_SECRET'] = '05075caea4f3cb63a0cebc5d65e446df4dfc9598932cf3ddc751deb8eee5baf3'

c.GitLabOAuthenticator.oauth_callback_url = os.environ['OAUTH_CALLBACK_URL']
c.GitlabOAuthenticator.client_id = os.environ['GITLAB_CLIENT_ID']
c.GitlabOAuthenticator.client_secret = os.environ['GITLAB_CLIENT_SECRET']


c.Authenticator.whitelist = whitelist = set()
c.Authenticator.admin_users = admin = set()

here = os.path.dirname(__file__)
with open(os.path.join(os.path.dirname(__file__), 'userlist')) as f:
    for line in f:
        if not line:
            continue
        parts = line.split()
        name = parts[0]
        whitelist.add(name)
        if len(parts) > 1 and parts[1] == 'admin':
            admin.add(name)

5、创建userlist文件并将其复制到volume

文件内容:

qw admin

这里只需要添加一个admin账户即可,因为其他账户后期都可以直接在界面中增加。

cp userlist /data/jupyterhub/userlist

6、build jupyterhub镜像

由于Docker中要用到pip,所以建议修改下pip源。新建pip.conf文件。内容为:

[global]
trusted-host=mirrors.aliyun.com
index-url=http://mirrors.aliyun.com/pypi/simple/

创建Dockerfile,文件内容为:

ARG BASE_IMAGE=jupyterhub/jupyterhub:1.0.0
FROM ${BASE_IMAGE}
ADD pip.conf /etc/pip.conf
RUN pip install --no-cache --upgrade jupyter
RUN pip install --no-cache dockerspawner
RUN pip install --no-cache oauthenticator
ENV GITLAB_HOST=http://git.domain.com
EXPOSE 8000

完成后执行:

docker build -t qw/jupyterhub .

7、build singleuser镜像(多用户支持)

创建Dockerfile,内容为:

ARG BASE_IMAGE=jupyterhub/singleuser:1.0.0
FROM ${BASE_IMAGE}

# 加速
ADD pip.conf /etc/pip.conf
RUN conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
RUN conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
RUN conda config --set show_channel_urls yes

# Install jupyterlab
RUN conda install -c conda-forge jupyterlab
# RUN pip install jupyterlab

RUN jupyter serverextension enable --py jupyterlab --sys-prefix
ENV GITLAB_HOST=http://git.domain.com

USER jovyan

完成后执行:

docker build -t qw/jupyter_lab_singleuser .

8、开启容器

docker run -d --name jupyterhub -p8000:8000 --network jupyterhub_network -v /var/run/docker.sock:/var/run/docker.sock -v /data/jupyterhub:/srv/jupyterhub qw/jupyterhub:latest

如报如下错误:

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/images/json?all=1: dial unix /var/run/docker.sock: connect: permission denied

则执行:

sudo chmod 666 /var/run/docker.sock

9、其他相关

Jupyterhub的配置文件中设置了共享目录,然是在实际使用时会报没有权限的问题。解决方案:

sudo chmod -R 777 /var/lib/docker/volumes/jupyterhub-shared

Docker调试相关命令

sudo service docker stop #关闭Docker服务
sudo rm -rf /var/lib/docker/ #删除所有Docker镜像
sudo service docker start #启动Docker服务

docker images -a #显示所有Docker镜像
docker rmi qw/jupyterhub #删除指定Docker镜像

docker container ls -a #显示所有docker容器
docker container stop jupyterhub #停止指定Docker容器
docker container start jupyterhub #开启指定Docker容器
docker container rm jupyterhub #删除指定Docker容器
docker logs --details jupyterhub #显示指定容器日志
docker exec -it jupyterhub bash # 进入指定容器(按Ctrl+D退出)



网友评论已有0条评论, 我也要评论

发表评论

*

* (保密)

Ctrl+Enter 快捷回复