← Older revision Revision as of 17:04, 15 January 2020 (25 intermediate revisions by the same user not shown)Line 1: Line 1:  ==About== ==About==  +Reference guides:  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-from-glacial-to-lightning-speed/ Supercharging Object Detection in Video: from Glacial to Lightning Speed]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-videos-setup/ Setup]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-first-app/ First App]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-optimizing-decoding-and-graph-feeding/ Optimizing Decoding and Graph Feeding]  +  +Target projects:  +* '''tensorflow-object-detection-cpp''' - contains a sample model, labels and a test video  +* '''fast_od''' - contains feeding Tensorflow from GPU     ==Setup== ==Setup==  +Kubuntu 18.04, CUDA 10.0 (10.0 is a requirement for Java version of TF 1.15 in Maven CDN):  + <font size=2>sudo apt install build-essential  + sudo apt install git curl  +  + mkdir -p ~/git/tf_cv_cu  + cd ~/git/tf_cv_cu  +  + '''# Download section'''  + <nowiki>git clone https://github.com/fierval/tensorflow-object-detection-cpp.git  +git clone https://github.com/fierval/fast_od.git  +wget https://github.com/Kitware/CMake/releases/download/v3.16.2/cmake-3.16.2.tar.gz  +wget https://github.com/bazelbuild/bazel/releases/download/0.25.2/bazel_0.25.2-linux-x86_64.deb  +wget https://github.com/tensorflow/tensorflow/archive/v1.15.0.tar.gz -O tensorflow-1.15.0.tar.gz  +wget https://github.com/opencv/opencv/archive/3.4.9.tar.gz -O opencv-3.4.9.tar.gz  +wget https://github.com/opencv/opencv_contrib/archive/3.4.9.tar.gz -O opencv_contrib-3.4.9.tar.gz</nowiki>  +  + '''# Unpack archives'''  + tar xzvf cmake-3.16.2.tar.gz  + tar xzvf tensorflow-1.15.0.tar.gz  + tar xzvf opencv-3.4.9.tar.gz  + tar xzvf opencv_contrib-3.4.9.tar.gz  +  + '''# Build & update cmake'''  + sudo apt install libssl-dev  + sudo apt install qt4-default  + cd cmake-3.16.2  + ./bootstrap -qt-gui  + make -j8  + sudo make install  +  + cmake -version  + cmake-gui -version  +  + '''# Install bazel'''  + sudo dpkg -i bazel_0.25.2-linux-x86_64.deb  + bazel version  +  + '''# Build TF'''  + cd ~/git/tf_cv_cu/tensorflow-1.15.0/  + tensorflow/contrib/makefile/download_dependencies.sh  + ./configure  + '''# pick all defaults except select '''yes''' for CUDA support'''  + bazel build //tensorflow:libtensorflow_cc.so  +  + '''# Install TF with headers'''  + sudo mkdir /usr/local/tensorflow  + sudo mkdir /usr/local/tensorflow/include  + sudo cp -r tensorflow/contrib/makefile/downloads/eigen/Eigen /usr/local/tensorflow/include/  + sudo cp -r tensorflow/contrib/makefile/downloads/eigen/unsupported /usr/local/tensorflow/include/  + sudo cp tensorflow/contrib/makefile/downloads/nsync/public/* /usr/local/tensorflow/include/  + sudo cp -r bazel-genfiles/tensorflow /usr/local/tensorflow/include/  + sudo cp -r tensorflow/cc /usr/local/tensorflow/include/tensorflow  + sudo cp -r tensorflow/core /usr/local/tensorflow/include/tensorflow  + sudo mkdir /usr/local/tensorflow/include/third_party  + sudo cp -r third_party/eigen3 /usr/local/tensorflow/include/third_party/  + sudo mkdir /usr/local/tensorflow/lib  + sudo cp bazel-bin/tensorflow/libtensorflow_*.so /usr/local/tensorflow/lib  + '''# extras, not mentioned in the reference guide'''  + sudo cp -r tensorflow/contrib/makefile/downloads/absl/absl /usr/local/tensorflow/include/  + sudo cp -r tensorflow/contrib/makefile/downloads/protobuf/src/google /usr/local/tensorflow/include/  + sudo cp -r tensorflow/stream_executor /usr/local/tensorflow/include/tensorflow/  + sudo cp bazel-bin/tensorflow/* /usr/local/tensorflow/lib   + sudo ln -sf /usr/local/tensorflow/lib/libtensorflow_framework.so.1.15.0 /usr/local/tensorflow/lib/libtensorflow_framework.so  +  + '''# Build & install OpenCV'''  + sudo apt install libgtkglext1 libgtkglext1-dev  + sudo apt install libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev  + sudo apt install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev  + # sudo apt install libjasper-dev '''# no such package in 18.04'''  +  + sudo cp ~/git/tf_cv_cu/fast_od/docker/nvcuvid/dynlink_* /usr/local/cuda/include  +  + cd ~/git/tf_cv_cu/opencv-3.4.9  + mkdir build; cd build  + '''# haven't tested this line yet. Did from cmake-gui'''  + cmake -D WITH_CUDA=ON \  +      -D OPENCV_EXTRA_MODULES_PATH=~/git/tf_cv_cu/opencv_contrib-3.4.9/modules \  +      -D BUILD_PERF_TESTS=OFF \  +      -D BUILD_TESTS=OFF \  +      -D BUILD_opencv_cudacodec=ON \  +      -D WITH_NVCUVID=ON \  +      -D WITH_GTK_2_X=ON \  +      ..  + make -j8  + sudo make install  + sudo ldconfig  +  + '''# tensorflow-object-detection-cpp'''  + cd ~/git/tf_cv_cu/tensorflow-object-detection-cpp  + mkdir build; cd build  + cmake ..  + make  + ./tf_detector_example  +  + '''# fast_od'''  + cd ~/git/tf_cv_cu/fast_od  + mkdir build; cd build  + cmake ..  + make  + '''# update paths paths to the model, labels and the video from tensorflow-object-detection-cpp in fast_od/run_tf.sh'''  + cd ..  + ./run_tf_elphel.sh 1</font> Oleg

‎Setup

← Older revision Revision as of 01:56, 15 January 2020 (23 intermediate revisions by the same user not shown)Line 1: Line 1:  ==About== ==About==  +Reference guides:  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-from-glacial-to-lightning-speed/ Supercharging Object Detection in Video: from Glacial to Lightning Speed]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-videos-setup/ Setup]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-first-app/ First App]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-optimizing-decoding-and-graph-feeding/ Optimizing Decoding and Graph Feeding]  +  +Target projects:  +* '''tensorflow-object-detection-cpp''' - contains a sample model, labels and a test video  +* '''fast_od''' - contains feeding Tensorflow from GPU     ==Setup== ==Setup==  +Kubuntu 18.04, CUDA 10.0 (10.0 is a requirement for Java version of TF 1.15 in Maven CDN):  + <font size=2>sudo apt install build-essential  + sudo apt install git curl  +  + mkdir -p ~/git/tf_cv_cu  + cd ~/git/tf_cv_cu  +  + '''# Download section'''  + <nowiki>git clone https://github.com/fierval/tensorflow-object-detection-cpp.git  +git clone https://github.com/fierval/fast_od.git  +wget https://github.com/Kitware/CMake/releases/download/v3.16.2/cmake-3.16.2.tar.gz  +wget https://github.com/bazelbuild/bazel/releases/download/0.25.2/bazel_0.25.2-linux-x86_64.deb  +wget https://github.com/tensorflow/tensorflow/archive/v1.15.0.tar.gz -O tensorflow-1.15.0.tar.gz  +wget https://github.com/opencv/opencv/archive/3.4.9.tar.gz -O opencv-3.4.9.tar.gz  +wget https://github.com/opencv/opencv_contrib/archive/3.4.9.tar.gz -O opencv_contrib-3.4.9.tar.gz</nowiki>  +  + '''# Unpack archives'''  + tar xzvf cmake-3.16.2.tar.gz  + tar xzvf tensorflow-1.15.0.tar.gz  + tar xzvf opencv-3.4.9.tar.gz  + tar xzvf opencv_contrib-3.4.9.tar.gz  +  + '''# Build & update cmake'''  + sudo apt install libssl-dev  + sudo apt install qt4-default  + cd cmake-3.16.2  + ./bootstrap -qt-gui  + make -j8  + sudo make install  +  + cmake -version  + cmake-gui -version  +  + '''# Install bazel'''  + sudo dpkg -i bazel_0.25.2-linux-x86_64.deb  + bazel version  +  + '''# Build TF'''  + cd ~/git/tf_cv_cu/tensorflow-1.15.0/  + tensorflow/contrib/makefile/download_dependencies.sh  + ./configure  + '''# pick all defaults except select '''yes''' for CUDA support'''  + bazel build //tensorflow:libtensorflow_cc.so  +  + '''# Install TF with headers'''  + sudo mkdir /usr/local/tensorflow  + sudo mkdir /usr/local/tensorflow/include  + sudo cp -r tensorflow/contrib/makefile/downloads/eigen/Eigen /usr/local/tensorflow/include/  + sudo cp -r tensorflow/contrib/makefile/downloads/eigen/unsupported /usr/local/tensorflow/include/  + sudo cp tensorflow/contrib/makefile/downloads/nsync/public/* /usr/local/tensorflow/include/  + sudo cp -r bazel-genfiles/tensorflow /usr/local/tensorflow/include/  + sudo cp -r tensorflow/cc /usr/local/tensorflow/include/tensorflow  + sudo cp -r tensorflow/core /usr/local/tensorflow/include/tensorflow  + sudo mkdir /usr/local/tensorflow/include/third_party  + sudo cp -r third_party/eigen3 /usr/local/tensorflow/include/third_party/  + sudo mkdir /usr/local/tensorflow/lib  + sudo cp bazel-bin/tensorflow/libtensorflow_*.so /usr/local/tensorflow/lib  + '''# extras, not mentioned in the reference guide'''  + sudo cp -r tensorflow/contrib/makefile/downloads/absl/absl /usr/local/tensorflow/include/  + sudo cp -r tensorflow/contrib/makefile/downloads/protobuf/src/google /usr/local/tensorflow/include/  + sudo cp -r tensorflow/stream_executor /usr/local/tensorflow/include/tensorflow/  + sudo cp bazel-bin/tensorflow/* /usr/local/tensorflow/lib   + sudo ln -sf /usr/local/tensorflow/lib/libtensorflow_framework.so.1.15.0 /usr/local/tensorflow/lib/libtensorflow_framework.so  +  + '''# Build & install OpenCV'''  + sudo apt install libgtkglext1 libgtkglext1-dev  + sudo apt install libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev  + sudo apt install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev  + # sudo apt install libjasper-dev '''# no such package in 18.04'''  +  + sudo cp ~/git/tf_cv_cu/fast_od/docker/nvcuvid/dynlink_* /usr/local/cuda/include  +  + cd ~/git/tf_cv_cu/opencv-3.4.9  + mkdir build; cd build  + '''# haven't tested this line yet. Did from cmake-gui'''  + cmake -D WITH_CUDA=ON \  +      -D OPENCV_EXTRA_MODULES_PATH=~/git/tf_cv_cu/opencv_contrib-3.4.9/modules \  +      -D BUILD_PERF_TESTS=OFF \  +      -D BUILD_TESTS=OFF \  +      -D BUILD_opencv_cudacodec=ON \  +      -D WITH_NVCUVID=ON \  +      -D WITH_GTK_2_X=ON \  +      ..  + make -j8  + sudo make install  + sudo ldconfig  + </font> Oleg

‎Setup

← Older revision Revision as of 01:42, 15 January 2020 (16 intermediate revisions by the same user not shown)Line 1: Line 1:  ==About== ==About==  +Reference guides:  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-from-glacial-to-lightning-speed/ Supercharging Object Detection in Video: from Glacial to Lightning Speed]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-videos-setup/ Setup]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-first-app/ First App]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-optimizing-decoding-and-graph-feeding/ Optimizing Decoding and Graph Feeding]  +  +Target projects:  +* '''tensorflow-object-detection-cpp''' - contains a sample model, labels and a test video  +* '''fast_od''' - contains feeding Tensorflow from GPU     ==Setup== ==Setup==  +Kubuntu 18.04, CUDA 10.0 (10.0 is a requirement for Java version of TF 1.15 in Maven CDN):  + <font size=2>mkdir -p ~/git/tf_cv_cu  + cd ~/git/tf_cv_cu  +  + '''# Download section'''  + <nowiki>git clone https://github.com/fierval/tensorflow-object-detection-cpp.git  +git clone https://github.com/fierval/fast_od.git  +wget https://github.com/Kitware/CMake/releases/download/v3.16.2/cmake-3.16.2.tar.gz  +wget https://github.com/bazelbuild/bazel/releases/download/0.25.2/bazel_0.25.2-linux-x86_64.deb  +wget https://github.com/tensorflow/tensorflow/archive/v1.15.0.tar.gz -O tensorflow-1.15.0.tar.gz  +wget https://github.com/opencv/opencv/archive/3.4.9.tar.gz -O opencv-3.4.9.tar.gz  +wget https://github.com/opencv/opencv_contrib/archive/3.4.9.tar.gz -O opencv_contrib-3.4.9.tar.gz</nowiki>  +  + '''# Unpack archives'''  + tar xzvf cmake-3.16.2.tar.gz  + tar xzvf tensorflow-1.15.0.tar.gz  + tar xzvf opencv-3.4.9.tar.gz  + tar xzvf opencv_contrib-3.4.9.tar.gz  +  + '''# Build & update cmake'''  + sudo apt install libssl-dev  + sudo apt install qt4-default  + cd cmake-3.16.2  + ./bootstrap -qt-gui  + make -j8  + sudo make install  +  + cmake -version  + cmake-gui -version  +  + '''# Install bazel'''  + sudo dpkg -i bazel_0.25.2-linux-x86_64.deb  + bazel version  +  + '''# Build TF'''  + cd ~/git/tf_cv_cu/tensorflow-1.15.0/  + tensorflow/contrib/makefile/download_dependencies.sh  + ./configure  + # pick all defaults except select '''yes''' for CUDA support  + bazel build //tensorflow:libtensorflow_cc.so  +  + '''# Build OpenCV'''  + sudo apt install libgtkglext1 libgtkglext1-dev  + sudo apt install build-essential  + sudo apt install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev  + sudo apt install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev  + # sudo apt install libjasper-dev # missing in Kubuntu 18.04  +  + sudo cp ~/git/tf_cv_cu/fast_od/docker/nvcuvid/dynlink_* /usr/local/cuda/include  +  + cd ~/git/tf_cv_cu/opencv-3.4.9  + mkdir build; cd build  + cmake -D WITH_CUDA=ON \  +      -D OPENCV_EXTRA_MODULES_PATH=~/git/tf_cv_cu/opencv_contrib-3.4.9/modules \  +      -D BUILD_PERF_TESTS=OFF \  +      -D BUILD_TESTS=OFF \  +      -D BUILD_opencv_cudacodec=ON \  +      -D WITH_NVCUVID=ON \  +      -D WITH_GTK_2_X=ON \  +      ..  + make -j8  + sudo make install  + sudo ldconfig  + </font> Oleg

← Older revision Revision as of 00:30, 15 January 2020 Line 1: Line 1:  ==About== ==About==  +Reference guides:  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-from-glacial-to-lightning-speed/ Supercharging Object Detection in Video: from Glacial to Lightning Speed]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-videos-setup/ Setup]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-first-app/ First App]  +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-optimizing-decoding-and-graph-feeding/ Optimizing Decoding and Graph Feeding]  +     ==Setup== ==Setup== Oleg

Created page with "==About== ==Setup=="

New page

==About==

==Setup== Oleg

‎Tensorflow and OpenCV building notes

← Older revision Revision as of 00:49, 8 January 2020 Line 249: Line 249:     ==Tensorflow and OpenCV building notes== ==Tensorflow and OpenCV building notes== −===Targets 1===+===Build 1===  # TF 1.15.0 # TF 1.15.0  # CUDA 10.0 and Toolkit and stuff # CUDA 10.0 and Toolkit and stuff Line 261: Line 261:    3. ./configure   3. ./configure    4.   4.  +  +===Build 2===  +# TF 1.13.1  +# CUDA 10.0 and Toolkit and stuff  +# OpenCV 3.4.9  +  +====TF 1.13.1====  +* Will build with Bazel 0.21.0 (installed from [https://github.com/bazelbuild/bazel/releases/tag/0.21.0 deb archive]) Oleg

Andrey.filippov changed visibility of 6 revisions on page File:333 hd setup.jpg: content hidden, edit summary hidden and username hidden vandalism

Andrey.filippov

‎Notes

← Older revision Revision as of 23:12, 7 January 2020 (One intermediate revision by the same user not shown)Line 1: Line 1: −==Requirements==+==OS==  * Kubuntu 16.04 LTS * Kubuntu 16.04 LTS  +  ==Setup (guide)== ==Setup (guide)==  Just follow: Just follow: Line 246: Line 247:    # Then open a browser:   # Then open a browser:    '''http://localhost:6006'''</font>   '''http://localhost:6006'''</font>  +  +==Tensorflow and OpenCV building notes==  +===Targets 1===  +# TF 1.15.0  +# CUDA 10.0 and Toolkit and stuff  +# OpenCV 3.4.9  +  +====TF 1.15.0====  +* Will build with Bazel 0.25.2 (installed from [https://github.com/bazelbuild/bazel/releases/tag/0.25.2 deb archive])  +* TF - downloaded as [https://github.com/tensorflow/tensorflow/releases/tag/v1.15.0 tensorflow-1.15.0.tar.gz]  + 1. Unpack  + 2. cd tensorflow-1.15.0  + 3. ./configure  + 4. Oleg

‎Requirements

← Older revision Revision as of 20:16, 7 January 2020 Line 1: Line 1: −==Requirements==+==OS==  * Kubuntu 16.04 LTS * Kubuntu 16.04 LTS  +  ==Setup (guide)== ==Setup (guide)==  Just follow: Just follow: Oleg

← Older revision Revision as of 23:19, 27 December 2019 (12 intermediate revisions by the same user not shown)Line 2: Line 2:  * Install Eclipse * Install Eclipse  * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] −<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>  +* TF version is pulled from pom.xml  +* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]  +* Get some image samples, provide paths  +* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:  +<font size='1'>  + <?xml version="1.0" encoding="UTF-8"?>  + <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">  + <properties>  +    <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>  +    <entry key="ADVANCED_MODE">True</entry>  +    <entry key="DCT_MODE">True</entry>  +    <entry key="MODE_3D">False</entry>  +    <entry key="GPU_MODE">True</entry>  +    <entry key="LWIR_MODE">True</entry>  + </properties>  +</font>  +* Updated pom.xml to TF 1.15 - package exists  +* Install cuDNN all 3 packages - runtime, dev and docs. Used docs to verify installation - built mnistCUDNN:  + https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html  +* I think TF 1.15 maven package was built for CUDA 10.0 driver, and so it whines when 10.2 is installed.  + <font size=1 color=red>2019-12-27 13:05:15.754656: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754756: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754860: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754970: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.755075: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.755178: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.762197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7  + 2019-12-27 13:05:15.762227: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.  + Skipping registering GPU devices...</font>  +   +* TF 1.15 and CUDA 10.0 require GPU compute capability = 6.0, GeForce GTX 750Ti is 5.0:  + <font color=red size=1>2019-12-27 14:22:17.475717: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /home/oleg/GIT/imagej-elphel/target/classes/trained_model  + 2019-12-27 14:22:17.477009: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }  + 2019-12-27 14:22:17.503393: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3392030000 Hz  + 2019-12-27 14:22:17.504196: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f610dba1a20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:  + 2019-12-27 14:22:17.504235: I tensorflow/compiler/xla/service/service.cc:176]  StreamExecutor device (0): Host, Default Version  + 2019-12-27 14:22:17.505378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1  + 2019-12-27 14:22:17.517647: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero  + 2019-12-27 14:22:17.518168: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:  + name: GeForce GTX 750 Ti major: 5 minor: 0 memoryClockRate(GHz): 1.1105  + pciBusID: 0000:01:00.0  + 2019-12-27 14:22:17.518385: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0  + 2019-12-27 14:22:17.519624: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0  + 2019-12-27 14:22:17.675574: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0  + 2019-12-27 14:22:17.716621: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0  + 2019-12-27 14:22:18.160070: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0  + 2019-12-27 14:22:18.439862: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0  + 2019-12-27 14:22:18.443378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7  + 2019-12-27 14:22:18.443483: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero  + 2019-12-27 14:22:18.444034: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero  + 2019-12-27 14:22:18.444510: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1700] Ignoring visible gpu device (device: 0, name: GeForce GTX 750 Ti, pci bus id: 0000:01:00.0, compute capability: 5.0) with  + '''Cuda compute capability 5.0. The minimum required Cuda capability is 6.0.'''  + 2019-12-27 14:22:18.498425: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:  + 2019-12-27 14:22:18.498455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0  + 2019-12-27 14:22:18.498463: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:  N  + 2019-12-27 14:22:18.504855: I tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.  + 2019-12-27 14:22:18.528948: I tensorflow/cc/saved_model/loader.cc:151] Running initialization op on SavedModel bundle at path: /home/oleg/GIT/imagej-elphel/target/classes/trained_model  + 2019-12-27 14:22:18.581034: I tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 1105321 microseconds.  +</font>  +* So, <font color='darkgreen'>'''TF1.15 + CUDA 10.0 might work with GeForce GTX 1080 Ti (compute capability 6.1)'''</font>  +TF Test button in Eyesis_Correction plugin worked with CUDA 10.0 (even with nvidia-smi showing CUDA 10.1 - it's probably not relevant to the libs used) Oleg

← Older revision Revision as of 21:55, 27 December 2019 (10 intermediate revisions by the same user not shown)Line 2: Line 2:  * Install Eclipse * Install Eclipse  * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] −<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>  +* TF version is pulled from pom.xml  +* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]  +* Get some image samples, provide paths  +* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:  +<font size='1'>  + <?xml version="1.0" encoding="UTF-8"?>  + <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">  + <properties>  +    <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>  +    <entry key="ADVANCED_MODE">True</entry>  +    <entry key="DCT_MODE">True</entry>  +    <entry key="MODE_3D">False</entry>  +    <entry key="GPU_MODE">True</entry>  +    <entry key="LWIR_MODE">True</entry>  + </properties>  +</font>  +* Updated pom.xml to TF 1.15 - package exists  +* Install cuDNN all 3 packages - runtime, dev and docs. Used docs to verify installation - built mnistCUDNN:  + https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html  +* I think TF 1.15 maven package was built for CUDA 10.0 driver, and so it whines when 10.2 is installed.  + <font size=1 color=red>2019-12-27 13:05:15.754656: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754756: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754860: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754970: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.755075: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.755178: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.762197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7  + 2019-12-27 13:05:15.762227: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.  + Skipping registering GPU devices...</font>  +   +* TF 1.15 and CUDA 10.0 require GPU compute capability = 6.0, GeForce GTX 750Ti is 5.0:  + <font color=red size=1>2019-12-27 14:22:17.475717: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /home/oleg/GIT/imagej-elphel/target/classes/trained_model  + 2019-12-27 14:22:17.477009: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }  + 2019-12-27 14:22:17.503393: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3392030000 Hz  + 2019-12-27 14:22:17.504196: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f610dba1a20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:  + 2019-12-27 14:22:17.504235: I tensorflow/compiler/xla/service/service.cc:176]  StreamExecutor device (0): Host, Default Version  + 2019-12-27 14:22:17.505378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1  + 2019-12-27 14:22:17.517647: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero  + 2019-12-27 14:22:17.518168: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:  + name: GeForce GTX 750 Ti major: 5 minor: 0 memoryClockRate(GHz): 1.1105  + pciBusID: 0000:01:00.0  + 2019-12-27 14:22:17.518385: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0  + 2019-12-27 14:22:17.519624: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0  + 2019-12-27 14:22:17.675574: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0  + 2019-12-27 14:22:17.716621: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0  + 2019-12-27 14:22:18.160070: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0  + 2019-12-27 14:22:18.439862: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0  + 2019-12-27 14:22:18.443378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7  + 2019-12-27 14:22:18.443483: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero  + 2019-12-27 14:22:18.444034: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero  + 2019-12-27 14:22:18.444510: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1700] Ignoring visible gpu device (device: 0, name: GeForce GTX 750 Ti, pci bus id: 0000:01:00.0, compute capability: 5.0) with  + '''Cuda compute capability 5.0. The minimum required Cuda capability is 6.0.'''  + 2019-12-27 14:22:18.498425: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:  + 2019-12-27 14:22:18.498455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0  + 2019-12-27 14:22:18.498463: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:  N  + 2019-12-27 14:22:18.504855: I tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.  + 2019-12-27 14:22:18.528948: I tensorflow/cc/saved_model/loader.cc:151] Running initialization op on SavedModel bundle at path: /home/oleg/GIT/imagej-elphel/target/classes/trained_model  + 2019-12-27 14:22:18.581034: I tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 1105321 microseconds.  +</font>  +* So, <font color='darkgreen'>'''TF1.15 + CUDA 10.0 might work with GeForce GTX 1080 Ti (compute capability 6.1)'''</font> Oleg

← Older revision Revision as of 20:32, 27 December 2019 (7 intermediate revisions by the same user not shown)Line 2: Line 2:  * Install Eclipse * Install Eclipse  * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] −<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>  +* TF version is pulled from pom.xml  +* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]  +* Get some image samples, provide paths  +* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:  +<font size='1'>  + <?xml version="1.0" encoding="UTF-8"?>  + <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">  + <properties>  +    <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>  +    <entry key="ADVANCED_MODE">True</entry>  +    <entry key="DCT_MODE">True</entry>  +    <entry key="MODE_3D">False</entry>  +    <entry key="GPU_MODE">True</entry>  +    <entry key="LWIR_MODE">True</entry>  + </properties>  +</font>  +* Updaed pom.xml to TF 1.15 - package exists  +* Install cuDNN all 3 packages - runtime, dev and docs. Used docs to verify installation - built mnistCUDNN:  + https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html  +* I think TF 1.15 maven package was built for CUDA 10.0 driver, and so it whines when 10.2 is installed.  + <font size=1 color=red>2019-12-27 13:05:15.754656: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754756: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754860: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754970: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.755075: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.755178: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.762197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7  + 2019-12-27 13:05:15.762227: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.  + Skipping registering GPU devices...</font> Oleg

← Older revision Revision as of 20:11, 27 December 2019 (6 intermediate revisions by the same user not shown)Line 2: Line 2:  * Install Eclipse * Install Eclipse  * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] −<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>  +* TF version is pulled from pom.xml  +* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]  +* Get some image samples, provide paths  +* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:  +<font size='1'>  + <?xml version="1.0" encoding="UTF-8"?>  + <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">  + <properties>  +    <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>  +    <entry key="ADVANCED_MODE">True</entry>  +    <entry key="DCT_MODE">True</entry>  +    <entry key="MODE_3D">False</entry>  +    <entry key="GPU_MODE">True</entry>  +    <entry key="LWIR_MODE">True</entry>  + </properties>  +</font>  +* Updaed pom.xml to TF 1.15 - package exists  +* Install cuDNN all 3 packages - runtime, dev and docs. Used docs to verify installation - built mnistCUDNN:  + https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html  +* I think TF 1.15 maven package was built for CUDA 10.0 driver  + <font size=1 color=red>2019-12-27 13:05:15.754656: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754756: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754860: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754970: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.755075: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.755178: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.762197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7  + 2019-12-27 13:05:15.762227: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.  + Skipping registering GPU devices...</font> Oleg

‎ImageJ plugin

← Older revision Revision as of 19:04, 27 December 2019 (2 intermediate revisions by the same user not shown)Line 2: Line 2:  * Install Eclipse * Install Eclipse  * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] −<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>  +* TF version is pulled from pom.xml  +* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]  +* Get some image samples, provide paths  +* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:  +<font size='1'>  + <?xml version="1.0" encoding="UTF-8"?>  + <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">  + <properties>  +    <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>  +    <entry key="ADVANCED_MODE">True</entry>  +    <entry key="DCT_MODE">True</entry>  +    <entry key="MODE_3D">False</entry>  +    <entry key="GPU_MODE">True</entry>  +    <entry key="LWIR_MODE">True</entry>  + </properties>  +</font> Oleg

Created page with "==ImageJ plugin== * Install Eclipse * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] <font color='tomato'>'''Note: if the project is updated/pull..."

New page

==ImageJ plugin==
* Install Eclipse
* Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel]
<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font> Oleg

‎Notes

← Older revision Revision as of 00:39, 24 December 2019 (2 intermediate revisions by the same user not shown)Line 177: Line 177:       −==Walkthrough for CUDA 10.2 (Dec 2019)==+==Setup walkthrough for CUDA 10.2 (Dec 2019)==     ===Install CUDA=== ===Install CUDA=== Line 222: Line 222:     * In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.   * In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.   −* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 already register it.+* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it.     ====Notes==== ====Notes==== Line 241: Line 241:  * How to run tensorboard from the container: * How to run tensorboard from the container:    <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]   <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here] −  # From the running container's command line (since it was run with 'bash' in the step above):+  # From the running container's command line (since it was run with 'bash' in the step above).  + # set a correct --logdir    root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15   root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15    # Then open a browser:   # Then open a browser:    '''http://localhost:6006'''</font>   '''http://localhost:6006'''</font> Oleg

‎Notes

← Older revision Revision as of 00:39, 24 December 2019 (35 intermediate revisions by the same user not shown)Line 176: Line 176:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker    −==Walkthrough for CUDA 10.2 (Dec 2019)==+   +==Setup walkthrough for CUDA 10.2 (Dec 2019)==  +   +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Will have to reboot  +   +===Docker===  +====Instructions====  +'''https://www.tensorflow.org/install/docker'''  +   +Quote:  + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).  +   +====Docker images====  +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/:  +{| class='wikitable'  +!TF version  +!Python major version  +!GPU support  +!NAME:TAG for Docker command  +|-  +|align='center'|1.15  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|2  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu'''  +|}  +   +====nvidia-docker====  +Somehow it was already installed.  +   +* Check NVIDIA docker version  + ~$ nvidia-docker version  +   +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it.  +   +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  +  + ~$ '''docker run \'''  +    '''-it \'''  +    '''--rm \'''  +    '''--name devtest \'''  +    '''-p 0.0.0.0:6006:6006 \'''  +    '''--mount type=bind,source="$(pwd)"/target,target=/app \'''  +    '''--gpus all \'''  +    <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \'''  +    '''bash'''</font>  +   +* How to run tensorboard from the container:  + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]  + # From the running container's command line (since it was run with 'bash' in the step above).  + # set a correct --logdir  + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15  + # Then open a browser:  + '''http://localhost:6006'''</font> Oleg

‎Notes

← Older revision Revision as of 00:39, 24 December 2019 (36 intermediate revisions by the same user not shown)Line 175: Line 175:    # Test 3: Run a local script (and include a local dir) in contatiner:   # Test 3: Run a local script (and include a local dir) in contatiner:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker  +  +  +==Setup walkthrough for CUDA 10.2 (Dec 2019)==  +  +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Will have to reboot  +  +===Docker===  +====Instructions====  +'''https://www.tensorflow.org/install/docker'''  +  +Quote:  + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).  +  +====Docker images====  +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/:  +{| class='wikitable'  +!TF version  +!Python major version  +!GPU support  +!NAME:TAG for Docker command  +|-  +|align='center'|1.15  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|2  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu'''  +|}  +  +====nvidia-docker====  +Somehow it was already installed.  +  +* Check NVIDIA docker version  + ~$ nvidia-docker version  +  +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it.  +  +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  +  + ~$ '''docker run \'''  +    '''-it \'''  +    '''--rm \'''  +    '''--name devtest \'''  +    '''-p 0.0.0.0:6006:6006 \'''  +    '''--mount type=bind,source="$(pwd)"/target,target=/app \'''  +    '''--gpus all \'''  +    <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \'''  +    '''bash'''</font>  +  +* How to run tensorboard from the container:  + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]  + # From the running container's command line (since it was run with 'bash' in the step above).  + # set a correct --logdir  + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15  + # Then open a browser:  + '''http://localhost:6006'''</font> Oleg

‎nvidia-docker

← Older revision Revision as of 00:02, 24 December 2019 (34 intermediate revisions by the same user not shown)Line 175: Line 175:    # Test 3: Run a local script (and include a local dir) in contatiner:   # Test 3: Run a local script (and include a local dir) in contatiner:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker  +  +  +==Walkthrough for CUDA 10.2 (Dec 2019)==  +  +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Will have to reboot  +  +===Docker===  +====Instructions====  +'''https://www.tensorflow.org/install/docker'''  +  +Quote:  + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).  +  +====Docker images====  +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/:  +{| class='wikitable'  +!TF version  +!Python major version  +!GPU support  +!NAME:TAG for Docker command  +|-  +|align='center'|1.15  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|2  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu'''  +|}  +  +====nvidia-docker====  +Somehow it was already installed.  +  +* Check NVIDIA docker version  + ~$ nvidia-docker version  +  +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it.  +  +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  +  + ~$ '''docker run \'''  +    '''-it \'''  +    '''--rm \'''  +    '''--name devtest \'''  +    '''-p 0.0.0.0:6006:6006 \'''  +    '''--mount type=bind,source="$(pwd)"/target,target=/app \'''  +    '''--gpus all \'''  +    <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \'''  +    '''bash'''</font>  +  +* How to run tensorboard from the container:  + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]  + # From the running container's command line (since it was run with 'bash' in the step above):  + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15  + # Then open a browser:  + '''http://localhost:6006'''</font> Oleg

‎Docker images

← Older revision Revision as of 22:49, 23 December 2019 (33 intermediate revisions by the same user not shown)Line 175: Line 175:    # Test 3: Run a local script (and include a local dir) in contatiner:   # Test 3: Run a local script (and include a local dir) in contatiner:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker  +  +  +==Walkthrough for CUDA 10.2 (Dec 2019)==  +  +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Will have to reboot  +  +===Docker===  +====Instructions====  +'''https://www.tensorflow.org/install/docker'''  +  +Quote:  + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).  +  +====Docker images====  +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/:  +{| class='wikitable'  +!TF version  +!Python major version  +!GPU support  +!NAME:TAG for Docker command  +|-  +|align='center'|1.15  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|2  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu'''  +|}  +  +====nvidia-docker====  +Somehow it was already installed.  +  +* Check NVIDIA docker version  + ~$ nvidia-docker version  +  +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 already register it.  +  +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  +  + ~$ '''docker run \'''  +    '''-it \'''  +    '''--rm \'''  +    '''--name devtest \'''  +    '''-p 0.0.0.0:6006:6006 \'''  +    '''--mount type=bind,source="$(pwd)"/target,target=/app \'''  +    '''--gpus all \'''  +    <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \'''  +    '''bash'''</font>  +  +* How to run tensorboard from the container:  + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]  + # From the running container's command line (since it was run with 'bash' in the step above):  + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15  + # Then open a browser:  + '''http://localhost:6006'''</font> Oleg