{"id":123996,"date":"2022-04-18T08:00:00","date_gmt":"2022-04-18T08:00:00","guid":{"rendered":"https:\/\/fedoramagazine.org\/?p=36283&preview=true&preview_id=36283"},"modified":"2022-04-18T08:00:00","modified_gmt":"2022-04-18T08:00:00","slug":"mlcube-and-podman","status":"publish","type":"post","link":"https:\/\/sickgaming.net\/blog\/2022\/04\/18\/mlcube-and-podman\/","title":{"rendered":"MLCube and Podman"},"content":{"rendered":"

MLCube<\/a> is a new open source container based infrastructure specification introduced to enable reproducibility in Python based machine learning workflows. It can utilize tools such as Podman<\/a>, Singularity<\/a> and Docker<\/a>. Execution on remote platforms is also supported. One of the chairs of the MLCommons Best Practices working group that is developing MLCube is Diane Feddema<\/a> from Red Hat. This introductory article explains how to run the hello world MLCube example<\/a> using Podman on Fedora Linux.<\/p>\n

<\/span> <\/p>\n

Yazan Monshed<\/a> has written a very helpful introduction to Podman on Fedora<\/a> which gives more details on some of the steps used here.<\/p>\n

First install the necessary dependencies.<\/p>\n

sudo dnf -y update\nsudo dnf -y install podman git virtualenv \\ policycoreutils-python-utils<\/pre>\n
Then, following the documentation, setup a virtual environment and get the example code. To ensure reproducibility, use a specific commit as the project is being actively improved.<\/p>\n
virtualenv -p python3 .\/env_mlcube source .\/env_mlcube\/bin\/activate\ngit clone https:\/\/github.com\/mlcommons\/mlcube_examples.git cd .\/mlcube_examples\/hello_world\ngit checkout 5fe69bd\npip install mlcube mlcube-docker\nmlcube describe<\/pre>\n
Now change the runner command from docker<\/em> to podman<\/em> by editing the file $HOME\/mlcube.yaml<\/kbd> so that the line<\/p>\n
docker: docker<\/pre>\n
becomes<\/p>\n
docker: podman<\/pre>\n
If you are on a computer with x86_64 architecture, you can get the container using<\/p>\n
mlcube configure --mlcube=. --platform=docker<\/pre>\n
You will see a number of options<\/p>\n
? Please select an image: \u25b8 registry.fedoraproject.org\/mlcommons\/hello_world:0.0.1 registry.access.redhat.com\/mlcommons\/hello_world:0.0.1 docker.io\/mlcommons\/hello_world:0.0.1 quay.io\/mlcommons\/hello_world:0.0.1<\/pre>\n
Choose docker.io\/mlcommons\/hello_world:0.0.1<\/kbd> to obtain the container.<\/p>\n
If you are not on a computer with x86_64 architecture, you will need to build the container. Change the file $HOME\/mlcube.yaml<\/kbd> so that the line<\/p>\n
build_strategy: pull<\/pre>\n
becomes<\/p>\n
build_strategy: auto<\/pre>\n
and then build the container using<\/p>\n
mlcube configure --mlcube=. --platform=docker<\/pre>\n
To run the tests, you may need to set SELinux permissions in the directories appropriately. You can check that SELinux is enabled by typing<\/p>\n
sudo sestatus<\/pre>\n
which should give you output similar to<\/p>\n
SELinux status: enabled\n...<\/pre>\n
Josphat Mutai<\/a>, Christopher Smart<\/a> and Daniel Walsh<\/a> explain that you need to be careful in setting appropriate SELinux policies for files used by containers. Here, you will allow the container to read and write to the workspace<\/em> directory.<\/p>\n
sudo semanage fcontext -a -t container_file_t \"$PWD\/workspace(\/.*)?\"\nsudo restorecon -Rv $PWD\/workspace<\/pre>\n
Now check the directory policy by checking that<\/p>\n
ls -Z<\/pre>\n
gives output similar to<\/p>\n
unconfined_u:object_r:user_home_t:s0 Dockerfile\nunconfined_u:object_r:user_home_t:s0 README.md\nunconfined_u:object_r:user_home_t:s0 mlcube.yaml\nunconfined_u:object_r:user_home_t:s0 requirements.txt\nunconfined_u:object_r:container_file_t:s0 workspace<\/pre>\n
Now run the example<\/p>\n
mlcube run --mlcube=. --task=hello --platform=docker\nmlcube run --mlcube=. --task=bye --platform=docker<\/pre>\n
Finally, check that the output<\/p>\n
cat workspace\/chats\/chat_with_alice.txt<\/pre>\n
has text similar to<\/p>\n
Hi, Alice! Nice to meet you.\nBye, Alice! It was great talking to you.<\/pre>\n
You can create your own MLCube as described here<\/a>. Contributions to the MLCube examples repository<\/a> are welcome. Udica<\/a> is a new project that promises more fine grained SELinux policy controls for containers that are easy for system administrators to apply. Active development of these projects is ongoing. Testing and providing feedback on them would help make secure data management on systems with SELinux easier and more effective.<\/p>\n","protected":false},"excerpt":{"rendered":"
MLCube is a new open source container based infrastructure specification introduced to enable reproducibility in Python based machine learning workflows. It can utilize tools such as Podman, Singularity and Docker. Execution on remote platforms is also supported. One of the chairs of the MLCommons Best Practices working group that is developing MLCube is Diane Feddema […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[48],"tags":[45,42,43,117,46,1325,47,540,542],"class_list":["post-123996","post","type-post","status-publish","format-standard","hentry","category-fedora-os","tag-fedora","tag-for-developers","tag-for-system-administrators","tag-machine-learning","tag-magazine","tag-mlcube","tag-news","tag-podman","tag-selinux"],"_links":{"self":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/123996","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/comments?post=123996"}],"version-history":[{"count":0,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/123996\/revisions"}],"wp:attachment":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media?parent=123996"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/categories?post=123996"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/tags?post=123996"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}