Feeding Tensorflow from GPU
For Java
← Older revision Revision as of 18:48, 20 February 2020 (One intermediate revision by the same user not shown)Line 113: Line 113: cd .. cd .. ./run_tf_elphel.sh 1</font> ./run_tf_elphel.sh 1</font> + +==For Java== +* https://www.tensorflow.org/install/lang_java which leads to a more detailed: +* https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/README.md + +In short, to Build TF for Java from source: + <font size=2>./configure + bazel build --config opt \ + //tensorflow/java:tensorflow \ + //tensorflow/java:libtensorflow_jni</font> OlegFeeding Tensorflow from GPU
← Older revision
Revision as of 17:04, 15 January 2020
(25 intermediate revisions by the same user not shown)Line 1:
Line 1:
==About== ==About==
+Reference guides:
+* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-from-glacial-to-lightning-speed/ Supercharging Object Detection in Video: from Glacial to Lightning Speed]
+* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-videos-setup/ Setup]
+* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-first-app/ First App]
+* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-optimizing-decoding-and-graph-feeding/ Optimizing Decoding and Graph Feeding]
+
+Target projects:
+* '''tensorflow-object-detection-cpp''' - contains a sample model, labels and a test video
+* '''fast_od''' - contains feeding Tensorflow from GPU
==Setup== ==Setup==
+Kubuntu 18.04, CUDA 10.0 (10.0 is a requirement for Java version of TF 1.15 in Maven CDN):
+ <font size=2>sudo apt install build-essential
+ sudo apt install git curl
+
+ mkdir -p ~/git/tf_cv_cu
+ cd ~/git/tf_cv_cu
+
+ '''# Download section'''
+ <nowiki>git clone https://github.com/fierval/tensorflow-object-detection-cpp.git
+git clone https://github.com/fierval/fast_od.git
+wget https://github.com/Kitware/CMake/releases/download/v3.16.2/cmake-3.16.2.tar.gz
+wget https://github.com/bazelbuild/bazel/releases/download/0.25.2/bazel_0.25.2-linux-x86_64.deb
+wget https://github.com/tensorflow/tensorflow/archive/v1.15.0.tar.gz -O tensorflow-1.15.0.tar.gz
+wget https://github.com/opencv/opencv/archive/3.4.9.tar.gz -O opencv-3.4.9.tar.gz
+wget https://github.com/opencv/opencv_contrib/archive/3.4.9.tar.gz -O opencv_contrib-3.4.9.tar.gz</nowiki>
+
+ '''# Unpack archives'''
+ tar xzvf cmake-3.16.2.tar.gz
+ tar xzvf tensorflow-1.15.0.tar.gz
+ tar xzvf opencv-3.4.9.tar.gz
+ tar xzvf opencv_contrib-3.4.9.tar.gz
+
+ '''# Build & update cmake'''
+ sudo apt install libssl-dev
+ sudo apt install qt4-default
+ cd cmake-3.16.2
+ ./bootstrap -qt-gui
+ make -j8
+ sudo make install
+
+ cmake -version
+ cmake-gui -version
+
+ '''# Install bazel'''
+ sudo dpkg -i bazel_0.25.2-linux-x86_64.deb
+ bazel version
+
+ '''# Build TF'''
+ cd ~/git/tf_cv_cu/tensorflow-1.15.0/
+ tensorflow/contrib/makefile/download_dependencies.sh
+ ./configure
+ '''# pick all defaults except select '''yes''' for CUDA support'''
+ bazel build //tensorflow:libtensorflow_cc.so
+
+ '''# Install TF with headers'''
+ sudo mkdir /usr/local/tensorflow
+ sudo mkdir /usr/local/tensorflow/include
+ sudo cp -r tensorflow/contrib/makefile/downloads/eigen/Eigen /usr/local/tensorflow/include/
+ sudo cp -r tensorflow/contrib/makefile/downloads/eigen/unsupported /usr/local/tensorflow/include/
+ sudo cp tensorflow/contrib/makefile/downloads/nsync/public/* /usr/local/tensorflow/include/
+ sudo cp -r bazel-genfiles/tensorflow /usr/local/tensorflow/include/
+ sudo cp -r tensorflow/cc /usr/local/tensorflow/include/tensorflow
+ sudo cp -r tensorflow/core /usr/local/tensorflow/include/tensorflow
+ sudo mkdir /usr/local/tensorflow/include/third_party
+ sudo cp -r third_party/eigen3 /usr/local/tensorflow/include/third_party/
+ sudo mkdir /usr/local/tensorflow/lib
+ sudo cp bazel-bin/tensorflow/libtensorflow_*.so /usr/local/tensorflow/lib
+ '''# extras, not mentioned in the reference guide'''
+ sudo cp -r tensorflow/contrib/makefile/downloads/absl/absl /usr/local/tensorflow/include/
+ sudo cp -r tensorflow/contrib/makefile/downloads/protobuf/src/google /usr/local/tensorflow/include/
+ sudo cp -r tensorflow/stream_executor /usr/local/tensorflow/include/tensorflow/
+ sudo cp bazel-bin/tensorflow/* /usr/local/tensorflow/lib
+ sudo ln -sf /usr/local/tensorflow/lib/libtensorflow_framework.so.1.15.0 /usr/local/tensorflow/lib/libtensorflow_framework.so
+
+ '''# Build & install OpenCV'''
+ sudo apt install libgtkglext1 libgtkglext1-dev
+ sudo apt install libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
+ sudo apt install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev
+ # sudo apt install libjasper-dev '''# no such package in 18.04'''
+
+ sudo cp ~/git/tf_cv_cu/fast_od/docker/nvcuvid/dynlink_* /usr/local/cuda/include
+
+ cd ~/git/tf_cv_cu/opencv-3.4.9
+ mkdir build; cd build
+ '''# haven't tested this line yet. Did from cmake-gui'''
+ cmake -D WITH_CUDA=ON \
+ -D OPENCV_EXTRA_MODULES_PATH=~/git/tf_cv_cu/opencv_contrib-3.4.9/modules \
+ -D BUILD_PERF_TESTS=OFF \
+ -D BUILD_TESTS=OFF \
+ -D BUILD_opencv_cudacodec=ON \
+ -D WITH_NVCUVID=ON \
+ -D WITH_GTK_2_X=ON \
+ ..
+ make -j8
+ sudo make install
+ sudo ldconfig
+
+ '''# tensorflow-object-detection-cpp'''
+ cd ~/git/tf_cv_cu/tensorflow-object-detection-cpp
+ mkdir build; cd build
+ cmake ..
+ make
+ ./tf_detector_example
+
+ '''# fast_od'''
+ cd ~/git/tf_cv_cu/fast_od
+ mkdir build; cd build
+ cmake ..
+ make
+ '''# update paths paths to the model, labels and the video from tensorflow-object-detection-cpp in fast_od/run_tf.sh'''
+ cd ..
+ ./run_tf_elphel.sh 1</font>
Oleg
Feeding Tensorflow from GPU
Setup
← Older revision Revision as of 01:56, 15 January 2020 (23 intermediate revisions by the same user not shown)Line 1: Line 1: ==About== ==About== +Reference guides: +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-from-glacial-to-lightning-speed/ Supercharging Object Detection in Video: from Glacial to Lightning Speed] +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-videos-setup/ Setup] +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-first-app/ First App] +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-optimizing-decoding-and-graph-feeding/ Optimizing Decoding and Graph Feeding] + +Target projects: +* '''tensorflow-object-detection-cpp''' - contains a sample model, labels and a test video +* '''fast_od''' - contains feeding Tensorflow from GPU ==Setup== ==Setup== +Kubuntu 18.04, CUDA 10.0 (10.0 is a requirement for Java version of TF 1.15 in Maven CDN): + <font size=2>sudo apt install build-essential + sudo apt install git curl + + mkdir -p ~/git/tf_cv_cu + cd ~/git/tf_cv_cu + + '''# Download section''' + <nowiki>git clone https://github.com/fierval/tensorflow-object-detection-cpp.git +git clone https://github.com/fierval/fast_od.git +wget https://github.com/Kitware/CMake/releases/download/v3.16.2/cmake-3.16.2.tar.gz +wget https://github.com/bazelbuild/bazel/releases/download/0.25.2/bazel_0.25.2-linux-x86_64.deb +wget https://github.com/tensorflow/tensorflow/archive/v1.15.0.tar.gz -O tensorflow-1.15.0.tar.gz +wget https://github.com/opencv/opencv/archive/3.4.9.tar.gz -O opencv-3.4.9.tar.gz +wget https://github.com/opencv/opencv_contrib/archive/3.4.9.tar.gz -O opencv_contrib-3.4.9.tar.gz</nowiki> + + '''# Unpack archives''' + tar xzvf cmake-3.16.2.tar.gz + tar xzvf tensorflow-1.15.0.tar.gz + tar xzvf opencv-3.4.9.tar.gz + tar xzvf opencv_contrib-3.4.9.tar.gz + + '''# Build & update cmake''' + sudo apt install libssl-dev + sudo apt install qt4-default + cd cmake-3.16.2 + ./bootstrap -qt-gui + make -j8 + sudo make install + + cmake -version + cmake-gui -version + + '''# Install bazel''' + sudo dpkg -i bazel_0.25.2-linux-x86_64.deb + bazel version + + '''# Build TF''' + cd ~/git/tf_cv_cu/tensorflow-1.15.0/ + tensorflow/contrib/makefile/download_dependencies.sh + ./configure + '''# pick all defaults except select '''yes''' for CUDA support''' + bazel build //tensorflow:libtensorflow_cc.so + + '''# Install TF with headers''' + sudo mkdir /usr/local/tensorflow + sudo mkdir /usr/local/tensorflow/include + sudo cp -r tensorflow/contrib/makefile/downloads/eigen/Eigen /usr/local/tensorflow/include/ + sudo cp -r tensorflow/contrib/makefile/downloads/eigen/unsupported /usr/local/tensorflow/include/ + sudo cp tensorflow/contrib/makefile/downloads/nsync/public/* /usr/local/tensorflow/include/ + sudo cp -r bazel-genfiles/tensorflow /usr/local/tensorflow/include/ + sudo cp -r tensorflow/cc /usr/local/tensorflow/include/tensorflow + sudo cp -r tensorflow/core /usr/local/tensorflow/include/tensorflow + sudo mkdir /usr/local/tensorflow/include/third_party + sudo cp -r third_party/eigen3 /usr/local/tensorflow/include/third_party/ + sudo mkdir /usr/local/tensorflow/lib + sudo cp bazel-bin/tensorflow/libtensorflow_*.so /usr/local/tensorflow/lib + '''# extras, not mentioned in the reference guide''' + sudo cp -r tensorflow/contrib/makefile/downloads/absl/absl /usr/local/tensorflow/include/ + sudo cp -r tensorflow/contrib/makefile/downloads/protobuf/src/google /usr/local/tensorflow/include/ + sudo cp -r tensorflow/stream_executor /usr/local/tensorflow/include/tensorflow/ + sudo cp bazel-bin/tensorflow/* /usr/local/tensorflow/lib + sudo ln -sf /usr/local/tensorflow/lib/libtensorflow_framework.so.1.15.0 /usr/local/tensorflow/lib/libtensorflow_framework.so + + '''# Build & install OpenCV''' + sudo apt install libgtkglext1 libgtkglext1-dev + sudo apt install libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev + sudo apt install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev + # sudo apt install libjasper-dev '''# no such package in 18.04''' + + sudo cp ~/git/tf_cv_cu/fast_od/docker/nvcuvid/dynlink_* /usr/local/cuda/include + + cd ~/git/tf_cv_cu/opencv-3.4.9 + mkdir build; cd build + '''# haven't tested this line yet. Did from cmake-gui''' + cmake -D WITH_CUDA=ON \ + -D OPENCV_EXTRA_MODULES_PATH=~/git/tf_cv_cu/opencv_contrib-3.4.9/modules \ + -D BUILD_PERF_TESTS=OFF \ + -D BUILD_TESTS=OFF \ + -D BUILD_opencv_cudacodec=ON \ + -D WITH_NVCUVID=ON \ + -D WITH_GTK_2_X=ON \ + .. + make -j8 + sudo make install + sudo ldconfig + </font> OlegFeeding Tensorflow from GPU
Setup
← Older revision Revision as of 01:42, 15 January 2020 (16 intermediate revisions by the same user not shown)Line 1: Line 1: ==About== ==About== +Reference guides: +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-from-glacial-to-lightning-speed/ Supercharging Object Detection in Video: from Glacial to Lightning Speed] +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-videos-setup/ Setup] +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-first-app/ First App] +* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-optimizing-decoding-and-graph-feeding/ Optimizing Decoding and Graph Feeding] + +Target projects: +* '''tensorflow-object-detection-cpp''' - contains a sample model, labels and a test video +* '''fast_od''' - contains feeding Tensorflow from GPU ==Setup== ==Setup== +Kubuntu 18.04, CUDA 10.0 (10.0 is a requirement for Java version of TF 1.15 in Maven CDN): + <font size=2>mkdir -p ~/git/tf_cv_cu + cd ~/git/tf_cv_cu + + '''# Download section''' + <nowiki>git clone https://github.com/fierval/tensorflow-object-detection-cpp.git +git clone https://github.com/fierval/fast_od.git +wget https://github.com/Kitware/CMake/releases/download/v3.16.2/cmake-3.16.2.tar.gz +wget https://github.com/bazelbuild/bazel/releases/download/0.25.2/bazel_0.25.2-linux-x86_64.deb +wget https://github.com/tensorflow/tensorflow/archive/v1.15.0.tar.gz -O tensorflow-1.15.0.tar.gz +wget https://github.com/opencv/opencv/archive/3.4.9.tar.gz -O opencv-3.4.9.tar.gz +wget https://github.com/opencv/opencv_contrib/archive/3.4.9.tar.gz -O opencv_contrib-3.4.9.tar.gz</nowiki> + + '''# Unpack archives''' + tar xzvf cmake-3.16.2.tar.gz + tar xzvf tensorflow-1.15.0.tar.gz + tar xzvf opencv-3.4.9.tar.gz + tar xzvf opencv_contrib-3.4.9.tar.gz + + '''# Build & update cmake''' + sudo apt install libssl-dev + sudo apt install qt4-default + cd cmake-3.16.2 + ./bootstrap -qt-gui + make -j8 + sudo make install + + cmake -version + cmake-gui -version + + '''# Install bazel''' + sudo dpkg -i bazel_0.25.2-linux-x86_64.deb + bazel version + + '''# Build TF''' + cd ~/git/tf_cv_cu/tensorflow-1.15.0/ + tensorflow/contrib/makefile/download_dependencies.sh + ./configure + # pick all defaults except select '''yes''' for CUDA support + bazel build //tensorflow:libtensorflow_cc.so + + '''# Build OpenCV''' + sudo apt install libgtkglext1 libgtkglext1-dev + sudo apt install build-essential + sudo apt install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev + sudo apt install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev + # sudo apt install libjasper-dev # missing in Kubuntu 18.04 + + sudo cp ~/git/tf_cv_cu/fast_od/docker/nvcuvid/dynlink_* /usr/local/cuda/include + + cd ~/git/tf_cv_cu/opencv-3.4.9 + mkdir build; cd build + cmake -D WITH_CUDA=ON \ + -D OPENCV_EXTRA_MODULES_PATH=~/git/tf_cv_cu/opencv_contrib-3.4.9/modules \ + -D BUILD_PERF_TESTS=OFF \ + -D BUILD_TESTS=OFF \ + -D BUILD_opencv_cudacodec=ON \ + -D WITH_NVCUVID=ON \ + -D WITH_GTK_2_X=ON \ + .. + make -j8 + sudo make install + sudo ldconfig + </font> OlegFeeding Tensorflow from GPU
← Older revision
Revision as of 00:30, 15 January 2020
Line 1:
Line 1:
==About== ==About==
+Reference guides:
+* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-from-glacial-to-lightning-speed/ Supercharging Object Detection in Video: from Glacial to Lightning Speed]
+* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-videos-setup/ Setup]
+* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-first-app/ First App]
+* [https://viralfsharp.com/2019/03/25/supercharging-object-detection-in-video-optimizing-decoding-and-graph-feeding/ Optimizing Decoding and Graph Feeding]
+
==Setup== ==Setup==
Oleg
Feeding Tensorflow from GPU
Created page with "==About== ==Setup=="
New page
==About====Setup== Oleg
Tensorflow with gpu
Tensorflow and OpenCV building notes
← Older revision Revision as of 00:49, 8 January 2020 Line 249: Line 249: ==Tensorflow and OpenCV building notes== ==Tensorflow and OpenCV building notes== −===Targets 1===+===Build 1=== # TF 1.15.0 # TF 1.15.0 # CUDA 10.0 and Toolkit and stuff # CUDA 10.0 and Toolkit and stuff Line 261: Line 261: 3. ./configure 3. ./configure 4. 4. + +===Build 2=== +# TF 1.13.1 +# CUDA 10.0 and Toolkit and stuff +# OpenCV 3.4.9 + +====TF 1.13.1==== +* Will build with Bazel 0.21.0 (installed from [https://github.com/bazelbuild/bazel/releases/tag/0.21.0 deb archive]) OlegFile:333 hd setup.jpg
Andrey.filippov changed visibility of 6 revisions on page File:333 hd setup.jpg: content hidden, edit summary hidden and username hidden vandalism
Andrey.filippovTensorflow with gpu
Notes
← Older revision Revision as of 23:12, 7 January 2020 (One intermediate revision by the same user not shown)Line 1: Line 1: −==Requirements==+==OS== * Kubuntu 16.04 LTS * Kubuntu 16.04 LTS + ==Setup (guide)== ==Setup (guide)== Just follow: Just follow: Line 246: Line 247: # Then open a browser: # Then open a browser: '''http://localhost:6006'''</font> '''http://localhost:6006'''</font> + +==Tensorflow and OpenCV building notes== +===Targets 1=== +# TF 1.15.0 +# CUDA 10.0 and Toolkit and stuff +# OpenCV 3.4.9 + +====TF 1.15.0==== +* Will build with Bazel 0.25.2 (installed from [https://github.com/bazelbuild/bazel/releases/tag/0.25.2 deb archive]) +* TF - downloaded as [https://github.com/tensorflow/tensorflow/releases/tag/v1.15.0 tensorflow-1.15.0.tar.gz] + 1. Unpack + 2. cd tensorflow-1.15.0 + 3. ./configure + 4. OlegTensorflow with gpu
Requirements
← Older revision Revision as of 20:16, 7 January 2020 Line 1: Line 1: −==Requirements==+==OS== * Kubuntu 16.04 LTS * Kubuntu 16.04 LTS + ==Setup (guide)== ==Setup (guide)== Just follow: Just follow: OlegQuad stereo tensorflow eclipse
← Older revision
Revision as of 23:19, 27 December 2019
(12 intermediate revisions by the same user not shown)Line 2:
Line 2:
* Install Eclipse * Install Eclipse
* Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel]
−<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>
+* TF version is pulled from pom.xml
+* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]
+* Get some image samples, provide paths
+* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:
+<font size='1'>
+ <?xml version="1.0" encoding="UTF-8"?>
+ <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
+ <properties>
+ <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>
+ <entry key="ADVANCED_MODE">True</entry>
+ <entry key="DCT_MODE">True</entry>
+ <entry key="MODE_3D">False</entry>
+ <entry key="GPU_MODE">True</entry>
+ <entry key="LWIR_MODE">True</entry>
+ </properties>
+</font>
+* Updated pom.xml to TF 1.15 - package exists
+* Install cuDNN all 3 packages - runtime, dev and docs. Used docs to verify installation - built mnistCUDNN:
+ https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html
+* I think TF 1.15 maven package was built for CUDA 10.0 driver, and so it whines when 10.2 is installed.
+ <font size=1 color=red>2019-12-27 13:05:15.754656: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754756: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754860: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754970: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.755075: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.755178: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.762197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
+ 2019-12-27 13:05:15.762227: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
+ Skipping registering GPU devices...</font>
+
+* TF 1.15 and CUDA 10.0 require GPU compute capability = 6.0, GeForce GTX 750Ti is 5.0:
+ <font color=red size=1>2019-12-27 14:22:17.475717: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /home/oleg/GIT/imagej-elphel/target/classes/trained_model
+ 2019-12-27 14:22:17.477009: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
+ 2019-12-27 14:22:17.503393: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3392030000 Hz
+ 2019-12-27 14:22:17.504196: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f610dba1a20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
+ 2019-12-27 14:22:17.504235: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
+ 2019-12-27 14:22:17.505378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
+ 2019-12-27 14:22:17.517647: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
+ 2019-12-27 14:22:17.518168: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
+ name: GeForce GTX 750 Ti major: 5 minor: 0 memoryClockRate(GHz): 1.1105
+ pciBusID: 0000:01:00.0
+ 2019-12-27 14:22:17.518385: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
+ 2019-12-27 14:22:17.519624: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
+ 2019-12-27 14:22:17.675574: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
+ 2019-12-27 14:22:17.716621: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
+ 2019-12-27 14:22:18.160070: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
+ 2019-12-27 14:22:18.439862: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
+ 2019-12-27 14:22:18.443378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
+ 2019-12-27 14:22:18.443483: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
+ 2019-12-27 14:22:18.444034: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
+ 2019-12-27 14:22:18.444510: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1700] Ignoring visible gpu device (device: 0, name: GeForce GTX 750 Ti, pci bus id: 0000:01:00.0, compute capability: 5.0) with
+ '''Cuda compute capability 5.0. The minimum required Cuda capability is 6.0.'''
+ 2019-12-27 14:22:18.498425: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
+ 2019-12-27 14:22:18.498455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
+ 2019-12-27 14:22:18.498463: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
+ 2019-12-27 14:22:18.504855: I tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.
+ 2019-12-27 14:22:18.528948: I tensorflow/cc/saved_model/loader.cc:151] Running initialization op on SavedModel bundle at path: /home/oleg/GIT/imagej-elphel/target/classes/trained_model
+ 2019-12-27 14:22:18.581034: I tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 1105321 microseconds.
+</font>
+* So, <font color='darkgreen'>'''TF1.15 + CUDA 10.0 might work with GeForce GTX 1080 Ti (compute capability 6.1)'''</font>
+TF Test button in Eyesis_Correction plugin worked with CUDA 10.0 (even with nvidia-smi showing CUDA 10.1 - it's probably not relevant to the libs used)
Oleg
Quad stereo tensorflow eclipse
← Older revision
Revision as of 21:55, 27 December 2019
(10 intermediate revisions by the same user not shown)Line 2:
Line 2:
* Install Eclipse * Install Eclipse
* Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel]
−<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>
+* TF version is pulled from pom.xml
+* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]
+* Get some image samples, provide paths
+* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:
+<font size='1'>
+ <?xml version="1.0" encoding="UTF-8"?>
+ <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
+ <properties>
+ <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>
+ <entry key="ADVANCED_MODE">True</entry>
+ <entry key="DCT_MODE">True</entry>
+ <entry key="MODE_3D">False</entry>
+ <entry key="GPU_MODE">True</entry>
+ <entry key="LWIR_MODE">True</entry>
+ </properties>
+</font>
+* Updated pom.xml to TF 1.15 - package exists
+* Install cuDNN all 3 packages - runtime, dev and docs. Used docs to verify installation - built mnistCUDNN:
+ https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html
+* I think TF 1.15 maven package was built for CUDA 10.0 driver, and so it whines when 10.2 is installed.
+ <font size=1 color=red>2019-12-27 13:05:15.754656: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754756: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754860: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754970: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.755075: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.755178: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.762197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
+ 2019-12-27 13:05:15.762227: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
+ Skipping registering GPU devices...</font>
+
+* TF 1.15 and CUDA 10.0 require GPU compute capability = 6.0, GeForce GTX 750Ti is 5.0:
+ <font color=red size=1>2019-12-27 14:22:17.475717: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /home/oleg/GIT/imagej-elphel/target/classes/trained_model
+ 2019-12-27 14:22:17.477009: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
+ 2019-12-27 14:22:17.503393: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3392030000 Hz
+ 2019-12-27 14:22:17.504196: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f610dba1a20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
+ 2019-12-27 14:22:17.504235: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
+ 2019-12-27 14:22:17.505378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
+ 2019-12-27 14:22:17.517647: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
+ 2019-12-27 14:22:17.518168: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
+ name: GeForce GTX 750 Ti major: 5 minor: 0 memoryClockRate(GHz): 1.1105
+ pciBusID: 0000:01:00.0
+ 2019-12-27 14:22:17.518385: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
+ 2019-12-27 14:22:17.519624: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
+ 2019-12-27 14:22:17.675574: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
+ 2019-12-27 14:22:17.716621: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
+ 2019-12-27 14:22:18.160070: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
+ 2019-12-27 14:22:18.439862: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
+ 2019-12-27 14:22:18.443378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
+ 2019-12-27 14:22:18.443483: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
+ 2019-12-27 14:22:18.444034: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
+ 2019-12-27 14:22:18.444510: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1700] Ignoring visible gpu device (device: 0, name: GeForce GTX 750 Ti, pci bus id: 0000:01:00.0, compute capability: 5.0) with
+ '''Cuda compute capability 5.0. The minimum required Cuda capability is 6.0.'''
+ 2019-12-27 14:22:18.498425: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
+ 2019-12-27 14:22:18.498455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
+ 2019-12-27 14:22:18.498463: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
+ 2019-12-27 14:22:18.504855: I tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.
+ 2019-12-27 14:22:18.528948: I tensorflow/cc/saved_model/loader.cc:151] Running initialization op on SavedModel bundle at path: /home/oleg/GIT/imagej-elphel/target/classes/trained_model
+ 2019-12-27 14:22:18.581034: I tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 1105321 microseconds.
+</font>
+* So, <font color='darkgreen'>'''TF1.15 + CUDA 10.0 might work with GeForce GTX 1080 Ti (compute capability 6.1)'''</font>
Oleg
Quad stereo tensorflow eclipse
← Older revision
Revision as of 20:32, 27 December 2019
(7 intermediate revisions by the same user not shown)Line 2:
Line 2:
* Install Eclipse * Install Eclipse
* Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel]
−<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>
+* TF version is pulled from pom.xml
+* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]
+* Get some image samples, provide paths
+* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:
+<font size='1'>
+ <?xml version="1.0" encoding="UTF-8"?>
+ <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
+ <properties>
+ <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>
+ <entry key="ADVANCED_MODE">True</entry>
+ <entry key="DCT_MODE">True</entry>
+ <entry key="MODE_3D">False</entry>
+ <entry key="GPU_MODE">True</entry>
+ <entry key="LWIR_MODE">True</entry>
+ </properties>
+</font>
+* Updaed pom.xml to TF 1.15 - package exists
+* Install cuDNN all 3 packages - runtime, dev and docs. Used docs to verify installation - built mnistCUDNN:
+ https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html
+* I think TF 1.15 maven package was built for CUDA 10.0 driver, and so it whines when 10.2 is installed.
+ <font size=1 color=red>2019-12-27 13:05:15.754656: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754756: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754860: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754970: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.755075: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.755178: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.762197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
+ 2019-12-27 13:05:15.762227: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
+ Skipping registering GPU devices...</font>
Oleg
Quad stereo tensorflow eclipse
← Older revision
Revision as of 20:11, 27 December 2019
(6 intermediate revisions by the same user not shown)Line 2:
Line 2:
* Install Eclipse * Install Eclipse
* Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel]
−<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>
+* TF version is pulled from pom.xml
+* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]
+* Get some image samples, provide paths
+* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:
+<font size='1'>
+ <?xml version="1.0" encoding="UTF-8"?>
+ <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
+ <properties>
+ <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>
+ <entry key="ADVANCED_MODE">True</entry>
+ <entry key="DCT_MODE">True</entry>
+ <entry key="MODE_3D">False</entry>
+ <entry key="GPU_MODE">True</entry>
+ <entry key="LWIR_MODE">True</entry>
+ </properties>
+</font>
+* Updaed pom.xml to TF 1.15 - package exists
+* Install cuDNN all 3 packages - runtime, dev and docs. Used docs to verify installation - built mnistCUDNN:
+ https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html
+* I think TF 1.15 maven package was built for CUDA 10.0 driver
+ <font size=1 color=red>2019-12-27 13:05:15.754656: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754756: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754860: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.754970: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.755075: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.755178: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory
+ 2019-12-27 13:05:15.762197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
+ 2019-12-27 13:05:15.762227: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
+ Skipping registering GPU devices...</font>
Oleg
Quad stereo tensorflow eclipse
ImageJ plugin
← Older revision Revision as of 19:04, 27 December 2019 (2 intermediate revisions by the same user not shown)Line 2: Line 2: * Install Eclipse * Install Eclipse * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] −<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font> +* TF version is pulled from pom.xml +* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip] +* Get some image samples, provide paths +* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml: +<font size='1'> + <?xml version="1.0" encoding="UTF-8"?> + <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd"> + <properties> + <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment> + <entry key="ADVANCED_MODE">True</entry> + <entry key="DCT_MODE">True</entry> + <entry key="MODE_3D">False</entry> + <entry key="GPU_MODE">True</entry> + <entry key="LWIR_MODE">True</entry> + </properties> +</font> OlegQuad stereo tensorflow eclipse
Created page with "==ImageJ plugin== * Install Eclipse * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] <font color='tomato'>'''Note: if the project is updated/pull..."
New page
==ImageJ plugin==* Install Eclipse
* Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel]
<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font> Oleg
Tensorflow with gpu
Notes
← Older revision Revision as of 00:39, 24 December 2019 (2 intermediate revisions by the same user not shown)Line 177: Line 177: −==Walkthrough for CUDA 10.2 (Dec 2019)==+==Setup walkthrough for CUDA 10.2 (Dec 2019)== ===Install CUDA=== ===Install CUDA=== Line 222: Line 222: * In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used. * In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used. −* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 already register it.+* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it. ====Notes==== ====Notes==== Line 241: Line 241: * How to run tensorboard from the container: * How to run tensorboard from the container: <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here] <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here] − # From the running container's command line (since it was run with 'bash' in the step above):+ # From the running container's command line (since it was run with 'bash' in the step above). + # set a correct --logdir root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt''' # remove --bind_all for TF 1.15 root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt''' # remove --bind_all for TF 1.15 # Then open a browser: # Then open a browser: '''http://localhost:6006'''</font> '''http://localhost:6006'''</font> OlegTensorflow with gpu
Notes
← Older revision Revision as of 00:39, 24 December 2019 (35 intermediate revisions by the same user not shown)Line 176: Line 176: https://www.tensorflow.org/install/docker https://www.tensorflow.org/install/docker −==Walkthrough for CUDA 10.2 (Dec 2019)==+ +==Setup walkthrough for CUDA 10.2 (Dec 2019)== + +===Install CUDA=== +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit]. +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01 +** Will have to reboot + +===Docker=== +====Instructions==== +'''https://www.tensorflow.org/install/docker''' + +Quote: + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed). + +====Docker images==== +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/: +{| class='wikitable' +!TF version +!Python major version +!GPU support +!NAME:TAG for Docker command +|- +|align='center'|1.15 +|align='center'|3 +|align='center'|yes +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3''' +|- +|align='center'|2.0.0+ +|align='center'|3 +|align='center'|yes +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3''' +|- +|align='center'|2.0.0+ +|align='center'|2 +|align='center'|yes +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu''' +|} + +====nvidia-docker==== +Somehow it was already installed. + +* Check NVIDIA docker version + ~$ nvidia-docker version + +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used. +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it. + +====Notes==== +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well: + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from + # to '''/app''' in the docker container + + ~$ '''docker run \''' + '''-it \''' + '''--rm \''' + '''--name devtest \''' + '''-p 0.0.0.0:6006:6006 \''' + '''--mount type=bind,source="$(pwd)"/target,target=/app \''' + '''--gpus all \''' + <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \''' + '''bash'''</font> + +* How to run tensorboard from the container: + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here] + # From the running container's command line (since it was run with 'bash' in the step above). + # set a correct --logdir + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt''' # remove --bind_all for TF 1.15 + # Then open a browser: + '''http://localhost:6006'''</font> OlegTensorflow with gpu
Notes
← Older revision Revision as of 00:39, 24 December 2019 (36 intermediate revisions by the same user not shown)Line 175: Line 175: # Test 3: Run a local script (and include a local dir) in contatiner: # Test 3: Run a local script (and include a local dir) in contatiner: https://www.tensorflow.org/install/docker https://www.tensorflow.org/install/docker + + +==Setup walkthrough for CUDA 10.2 (Dec 2019)== + +===Install CUDA=== +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit]. +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01 +** Will have to reboot + +===Docker=== +====Instructions==== +'''https://www.tensorflow.org/install/docker''' + +Quote: + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed). + +====Docker images==== +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/: +{| class='wikitable' +!TF version +!Python major version +!GPU support +!NAME:TAG for Docker command +|- +|align='center'|1.15 +|align='center'|3 +|align='center'|yes +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3''' +|- +|align='center'|2.0.0+ +|align='center'|3 +|align='center'|yes +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3''' +|- +|align='center'|2.0.0+ +|align='center'|2 +|align='center'|yes +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu''' +|} + +====nvidia-docker==== +Somehow it was already installed. + +* Check NVIDIA docker version + ~$ nvidia-docker version + +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used. +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it. + +====Notes==== +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well: + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from + # to '''/app''' in the docker container + + ~$ '''docker run \''' + '''-it \''' + '''--rm \''' + '''--name devtest \''' + '''-p 0.0.0.0:6006:6006 \''' + '''--mount type=bind,source="$(pwd)"/target,target=/app \''' + '''--gpus all \''' + <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \''' + '''bash'''</font> + +* How to run tensorboard from the container: + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here] + # From the running container's command line (since it was run with 'bash' in the step above). + # set a correct --logdir + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt''' # remove --bind_all for TF 1.15 + # Then open a browser: + '''http://localhost:6006'''</font> OlegTensorflow with gpu
nvidia-docker
← Older revision Revision as of 00:02, 24 December 2019 (34 intermediate revisions by the same user not shown)Line 175: Line 175: # Test 3: Run a local script (and include a local dir) in contatiner: # Test 3: Run a local script (and include a local dir) in contatiner: https://www.tensorflow.org/install/docker https://www.tensorflow.org/install/docker + + +==Walkthrough for CUDA 10.2 (Dec 2019)== + +===Install CUDA=== +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit]. +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01 +** Will have to reboot + +===Docker=== +====Instructions==== +'''https://www.tensorflow.org/install/docker''' + +Quote: + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed). + +====Docker images==== +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/: +{| class='wikitable' +!TF version +!Python major version +!GPU support +!NAME:TAG for Docker command +|- +|align='center'|1.15 +|align='center'|3 +|align='center'|yes +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3''' +|- +|align='center'|2.0.0+ +|align='center'|3 +|align='center'|yes +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3''' +|- +|align='center'|2.0.0+ +|align='center'|2 +|align='center'|yes +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu''' +|} + +====nvidia-docker==== +Somehow it was already installed. + +* Check NVIDIA docker version + ~$ nvidia-docker version + +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used. +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it. + +====Notes==== +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well: + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from + # to '''/app''' in the docker container + + ~$ '''docker run \''' + '''-it \''' + '''--rm \''' + '''--name devtest \''' + '''-p 0.0.0.0:6006:6006 \''' + '''--mount type=bind,source="$(pwd)"/target,target=/app \''' + '''--gpus all \''' + <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \''' + '''bash'''</font> + +* How to run tensorboard from the container: + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here] + # From the running container's command line (since it was run with 'bash' in the step above): + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt''' # remove --bind_all for TF 1.15 + # Then open a browser: + '''http://localhost:6006'''</font> Oleg