宿主服务器使用的是Ubuntu 18.04,需要注意的是Docker目前不支持Ubuntu 19.10。如要在19.10中使用Docker需要在Docker源配置时设置Ubuntu 18.04的版本标识:bionic。
deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable
Docker的安装
Docker的安装流程非常的简单,按以下命令执行即可:
sudo apt remove docker docker-engine docker.io sudo apt install apt-transport-https ca-certificates curl software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add – sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" sudo apt update sudo apt install docker-ce sudo usermod -aG docker $USER #将当前用户加入到Docker组 sudo echo "DOCKER_OPTS="--registry-mirror=https://registry.docker-cn.com"" >> /etc/default/docker #更改为国内源 sudo service docker restart
参考链接:
- https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-18-04
- https://www.linode.com/docs/applications/containers/install-docker-ce-ubuntu-1804/
Docker中使用GPU
安装好了普通的Docker以后,如果想在容器内使用GPU会非常麻烦,好在Nvidia为了让大家能在容器中愉快使用GPU,基于Docker开发了Nvidia-Docker,使得在容器中深度学习框架调用GPU变得极为容易
参考链接:https://github.com/NVIDIA/nvidia-docker
多用户jupyterhub的安装
1、拉取相关镜像:
不用拉取latest版本,latest版本存在Bug,安装完成后不能正常运行。别问我怎么知道的,血与泪~
docker pull jupyterhub/jupyterhub:1.0.0 docker pull jupyterhub/singleuser:1.0.0
2、创建jupyterhub_network网络
docker network create --driver bridge jupyterhub_network
3、创建jupyterhub的volume
sudo mkdir -pv /data/jupyterhub sudo chown -R root /data/jupyterhub sudo chmod -R 777 /data/jupyterhub
4、创建jupyterhub_config.py文件并将其复制到volume
cp jupyterhub_config.py /data/jupyterhub/jupyterhub_config.py
文件内容:
# Configuration file for Jupyter Hub c = get_config() # spawn with Docker c.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner' # Spawn containers from this image c.DockerSpawner.image = 'qw/jupyter_lab_singleuser:latest' # JupyterHub requires a single-user instance of the Notebook server, so we # default to using the `start-singleuser.sh` script included in the # jupyter/docker-stacks *-notebook images as the Docker run command when # spawning containers. Optionally, you can override the Docker run command # using the DOCKER_SPAWN_CMD environment variable. c.DockerSpawner.extra_create_kwargs.update({ 'command': "start-singleuser.sh --SingleUserNotebookApp.default_url=/lab" }) # Connect containers to this Docker network network_name = 'jupyterhub_network' c.DockerSpawner.use_internal_ip = True c.DockerSpawner.network_name = network_name # Pass the network name as argument to spawned containers c.DockerSpawner.extra_host_config = { 'network_mode': network_name } # Explicitly set notebook directory because we'll be mounting a host volume to # it. Most jupyter/docker-stacks *-notebook images run the Notebook server as # user `jovyan`, and set the notebook directory to `/home/jovyan/work`. # We follow the same convention. notebook_dir = '/home/jovyan' # notebook_dir = '/home/jovyan/work' c.DockerSpawner.notebook_dir = notebook_dir # Mount the real user's Docker volume on the host to the notebook user's # notebook directory in the container c.DockerSpawner.volumes = { 'jupyterhub-user-{username}': notebook_dir,'jupyterhub-shared': {"bind": '/home/jovyan/shared', "mode": "rw"}} # volume_driver is no longer a keyword argument to create_container() # c.DockerSpawner.extra_create_kwargs.update({ 'volume_driver': 'local' }) # Remove containers once they are stopped c.DockerSpawner.remove_containers = True # For debugging arguments passed to spawned containers c.DockerSpawner.debug = True # The docker instances need access to the Hub, so the default loopback port doesn't work: # from jupyter_client.localinterfaces import public_ips # c.JupyterHub.hub_ip = public_ips()[0] c.JupyterHub.hub_ip = 'jupyterhub' # IP Configurations c.JupyterHub.ip = '0.0.0.0' c.JupyterHub.port = 8000 # OAuth with GitLab import os c.JupyterHub.authenticator_class = 'oauthenticator.gitlab.GitLabOAuthenticator' os.environ['OAUTH_CALLBACK_URL'] = 'http://10.101.14.13:8000/hub/oauth_callback' os.environ['GITLAB_CLIENT_ID'] = 'd89d76ef002100f217f4a7c1fc73011ca4d9eee7bb5ff8ce3e9532ba7721e29e' os.environ['GITLAB_CLIENT_SECRET'] = '05075caea4f3cb63a0cebc5d65e446df4dfc9598932cf3ddc751deb8eee5baf3' c.GitLabOAuthenticator.oauth_callback_url = os.environ['OAUTH_CALLBACK_URL'] c.GitlabOAuthenticator.client_id = os.environ['GITLAB_CLIENT_ID'] c.GitlabOAuthenticator.client_secret = os.environ['GITLAB_CLIENT_SECRET'] c.Authenticator.whitelist = whitelist = set() c.Authenticator.admin_users = admin = set() here = os.path.dirname(__file__) with open(os.path.join(os.path.dirname(__file__), 'userlist')) as f: for line in f: if not line: continue parts = line.split() name = parts[0] whitelist.add(name) if len(parts) > 1 and parts[1] == 'admin': admin.add(name)
5、创建userlist文件并将其复制到volume
文件内容:
qw admin
这里只需要添加一个admin账户即可,因为其他账户后期都可以直接在界面中增加。
cp userlist /data/jupyterhub/userlist
6、build jupyterhub镜像
由于Docker中要用到pip,所以建议修改下pip源。新建pip.conf文件。内容为:
[global] trusted-host=mirrors.aliyun.com index-url=http://mirrors.aliyun.com/pypi/simple/
创建Dockerfile,文件内容为:
ARG BASE_IMAGE=jupyterhub/jupyterhub:1.0.0 FROM ${BASE_IMAGE} ADD pip.conf /etc/pip.conf RUN pip install --no-cache --upgrade jupyter RUN pip install --no-cache dockerspawner RUN pip install --no-cache oauthenticator ENV GITLAB_HOST=http://git.domain.com EXPOSE 8000
完成后执行:
docker build -t qw/jupyterhub .
7、build singleuser镜像(多用户支持)
创建Dockerfile,内容为:
ARG BASE_IMAGE=jupyterhub/singleuser:1.0.0 FROM ${BASE_IMAGE} # 加速 ADD pip.conf /etc/pip.conf RUN conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/ RUN conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/ RUN conda config --set show_channel_urls yes # Install jupyterlab RUN conda install -c conda-forge jupyterlab # RUN pip install jupyterlab RUN jupyter serverextension enable --py jupyterlab --sys-prefix ENV GITLAB_HOST=http://git.domain.com USER jovyan
完成后执行:
docker build -t qw/jupyter_lab_singleuser .
8、开启容器
docker run -d --name jupyterhub -p8000:8000 --network jupyterhub_network -v /var/run/docker.sock:/var/run/docker.sock -v /data/jupyterhub:/srv/jupyterhub qw/jupyterhub:latest
如报如下错误:
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/images/json?all=1: dial unix /var/run/docker.sock: connect: permission denied
则执行:
sudo chmod 666 /var/run/docker.sock
9、其他相关
Jupyterhub的配置文件中设置了共享目录,然是在实际使用时会报没有权限的问题。解决方案:
sudo chmod -R 777 /var/lib/docker/volumes/jupyterhub-shared
Docker调试相关命令
sudo service docker stop #关闭Docker服务 sudo rm -rf /var/lib/docker/ #删除所有Docker镜像 sudo service docker start #启动Docker服务 docker images -a #显示所有Docker镜像 docker rmi qw/jupyterhub #删除指定Docker镜像 docker container ls -a #显示所有docker容器 docker container stop jupyterhub #停止指定Docker容器 docker container start jupyterhub #开启指定Docker容器 docker container rm jupyterhub #删除指定Docker容器 docker logs --details jupyterhub #显示指定容器日志 docker exec -it jupyterhub bash # 进入指定容器(按Ctrl+D退出)
网友评论已有0条评论, 我也要评论