← Older revision Revision as of 20:11, 27 December 2019 (6 intermediate revisions by the same user not shown)Line 2: Line 2:  * Install Eclipse * Install Eclipse  * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] −<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>  +* TF version is pulled from pom.xml  +* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]  +* Get some image samples, provide paths  +* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:  +<font size='1'>  + <?xml version="1.0" encoding="UTF-8"?>  + <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">  + <properties>  +    <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>  +    <entry key="ADVANCED_MODE">True</entry>  +    <entry key="DCT_MODE">True</entry>  +    <entry key="MODE_3D">False</entry>  +    <entry key="GPU_MODE">True</entry>  +    <entry key="LWIR_MODE">True</entry>  + </properties>  +</font>  +* Updaed pom.xml to TF 1.15 - package exists  +* Install cuDNN all 3 packages - runtime, dev and docs. Used docs to verify installation - built mnistCUDNN:  + https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html  +* I think TF 1.15 maven package was built for CUDA 10.0 driver  + <font size=1 color=red>2019-12-27 13:05:15.754656: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754756: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754860: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.754970: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.755075: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.755178: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory  + 2019-12-27 13:05:15.762197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7  + 2019-12-27 13:05:15.762227: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.  + Skipping registering GPU devices...</font> Oleg

‎ImageJ plugin

← Older revision Revision as of 19:04, 27 December 2019 (2 intermediate revisions by the same user not shown)Line 2: Line 2:  * Install Eclipse * Install Eclipse  * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] −<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font>+<font color='red'>'''NOTE: if project is updated/pulled outside Eclipse - might need a manual refresh'''</font>  +* TF version is pulled from pom.xml  +* Trained TF model for EO sensors is auto-downloaded - [https://community.elphel.com/files/quad-stereo/ml/trained_model_v1.0.zip trained_model_v1.0.zip]  +* Get some image samples, provide paths  +* Before running the plugin (Eyesis_Correction), copy imagej options to /home/user/.imagejs/Eyesis_Correction.xml:  +<font size='1'>  + <?xml version="1.0" encoding="UTF-8"?>  + <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">  + <properties>  +    <comment>last updated Thu Sep 08 14:09:47 MDT 2042</comment>  +    <entry key="ADVANCED_MODE">True</entry>  +    <entry key="DCT_MODE">True</entry>  +    <entry key="MODE_3D">False</entry>  +    <entry key="GPU_MODE">True</entry>  +    <entry key="LWIR_MODE">True</entry>  + </properties>  +</font> Oleg

Created page with "==ImageJ plugin== * Install Eclipse * Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel] <font color='tomato'>'''Note: if the project is updated/pull..."

New page

==ImageJ plugin==
* Install Eclipse
* Clone and Import [https://git.elphel.com/Elphel/imagej-elphel imagej-elphel]
<font color='tomato'>'''Note: if the project is updated/pulled outside Eclipse - need manual refresh'''</font> Oleg

AndreyFilippov committed changes to the Elphel git project :
debugging disparity grid distortion calculation

‎Notes

← Older revision Revision as of 00:39, 24 December 2019 (2 intermediate revisions by the same user not shown)Line 177: Line 177:       −==Walkthrough for CUDA 10.2 (Dec 2019)==+==Setup walkthrough for CUDA 10.2 (Dec 2019)==     ===Install CUDA=== ===Install CUDA=== Line 222: Line 222:     * In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.   * In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.   −* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 already register it.+* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it.     ====Notes==== ====Notes==== Line 241: Line 241:  * How to run tensorboard from the container: * How to run tensorboard from the container:    <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]   <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here] −  # From the running container's command line (since it was run with 'bash' in the step above):+  # From the running container's command line (since it was run with 'bash' in the step above).  + # set a correct --logdir    root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15   root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15    # Then open a browser:   # Then open a browser:    '''http://localhost:6006'''</font>   '''http://localhost:6006'''</font> Oleg

‎Notes

← Older revision Revision as of 00:39, 24 December 2019 (35 intermediate revisions by the same user not shown)Line 176: Line 176:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker    −==Walkthrough for CUDA 10.2 (Dec 2019)==+   +==Setup walkthrough for CUDA 10.2 (Dec 2019)==  +   +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Will have to reboot  +   +===Docker===  +====Instructions====  +'''https://www.tensorflow.org/install/docker'''  +   +Quote:  + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).  +   +====Docker images====  +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/:  +{| class='wikitable'  +!TF version  +!Python major version  +!GPU support  +!NAME:TAG for Docker command  +|-  +|align='center'|1.15  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|2  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu'''  +|}  +   +====nvidia-docker====  +Somehow it was already installed.  +   +* Check NVIDIA docker version  + ~$ nvidia-docker version  +   +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it.  +   +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  +  + ~$ '''docker run \'''  +    '''-it \'''  +    '''--rm \'''  +    '''--name devtest \'''  +    '''-p 0.0.0.0:6006:6006 \'''  +    '''--mount type=bind,source="$(pwd)"/target,target=/app \'''  +    '''--gpus all \'''  +    <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \'''  +    '''bash'''</font>  +   +* How to run tensorboard from the container:  + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]  + # From the running container's command line (since it was run with 'bash' in the step above).  + # set a correct --logdir  + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15  + # Then open a browser:  + '''http://localhost:6006'''</font> Oleg

‎Notes

← Older revision Revision as of 00:39, 24 December 2019 (36 intermediate revisions by the same user not shown)Line 175: Line 175:    # Test 3: Run a local script (and include a local dir) in contatiner:   # Test 3: Run a local script (and include a local dir) in contatiner:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker  +  +  +==Setup walkthrough for CUDA 10.2 (Dec 2019)==  +  +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Will have to reboot  +  +===Docker===  +====Instructions====  +'''https://www.tensorflow.org/install/docker'''  +  +Quote:  + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).  +  +====Docker images====  +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/:  +{| class='wikitable'  +!TF version  +!Python major version  +!GPU support  +!NAME:TAG for Docker command  +|-  +|align='center'|1.15  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|2  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu'''  +|}  +  +====nvidia-docker====  +Somehow it was already installed.  +  +* Check NVIDIA docker version  + ~$ nvidia-docker version  +  +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it.  +  +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  +  + ~$ '''docker run \'''  +    '''-it \'''  +    '''--rm \'''  +    '''--name devtest \'''  +    '''-p 0.0.0.0:6006:6006 \'''  +    '''--mount type=bind,source="$(pwd)"/target,target=/app \'''  +    '''--gpus all \'''  +    <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \'''  +    '''bash'''</font>  +  +* How to run tensorboard from the container:  + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]  + # From the running container's command line (since it was run with 'bash' in the step above).  + # set a correct --logdir  + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15  + # Then open a browser:  + '''http://localhost:6006'''</font> Oleg

‎nvidia-docker

← Older revision Revision as of 00:02, 24 December 2019 (34 intermediate revisions by the same user not shown)Line 175: Line 175:    # Test 3: Run a local script (and include a local dir) in contatiner:   # Test 3: Run a local script (and include a local dir) in contatiner:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker  +  +  +==Walkthrough for CUDA 10.2 (Dec 2019)==  +  +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Will have to reboot  +  +===Docker===  +====Instructions====  +'''https://www.tensorflow.org/install/docker'''  +  +Quote:  + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).  +  +====Docker images====  +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/:  +{| class='wikitable'  +!TF version  +!Python major version  +!GPU support  +!NAME:TAG for Docker command  +|-  +|align='center'|1.15  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|2  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu'''  +|}  +  +====nvidia-docker====  +Somehow it was already installed.  +  +* Check NVIDIA docker version  + ~$ nvidia-docker version  +  +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 should have already registered it.  +  +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  +  + ~$ '''docker run \'''  +    '''-it \'''  +    '''--rm \'''  +    '''--name devtest \'''  +    '''-p 0.0.0.0:6006:6006 \'''  +    '''--mount type=bind,source="$(pwd)"/target,target=/app \'''  +    '''--gpus all \'''  +    <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \'''  +    '''bash'''</font>  +  +* How to run tensorboard from the container:  + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]  + # From the running container's command line (since it was run with 'bash' in the step above):  + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15  + # Then open a browser:  + '''http://localhost:6006'''</font> Oleg

‎Docker images

← Older revision Revision as of 22:49, 23 December 2019 (33 intermediate revisions by the same user not shown)Line 175: Line 175:    # Test 3: Run a local script (and include a local dir) in contatiner:   # Test 3: Run a local script (and include a local dir) in contatiner:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker  +  +  +==Walkthrough for CUDA 10.2 (Dec 2019)==  +  +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Will have to reboot  +  +===Docker===  +====Instructions====  +'''https://www.tensorflow.org/install/docker'''  +  +Quote:  + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).  +  +====Docker images====  +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/:  +{| class='wikitable'  +!TF version  +!Python major version  +!GPU support  +!NAME:TAG for Docker command  +|-  +|align='center'|1.15  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|2  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu'''  +|}  +  +====nvidia-docker====  +Somehow it was already installed.  +  +* Check NVIDIA docker version  + ~$ nvidia-docker version  +  +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 already register it.  +  +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  +  + ~$ '''docker run \'''  +    '''-it \'''  +    '''--rm \'''  +    '''--name devtest \'''  +    '''-p 0.0.0.0:6006:6006 \'''  +    '''--mount type=bind,source="$(pwd)"/target,target=/app \'''  +    '''--gpus all \'''  +    <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \'''  +    '''bash'''</font>  +  +* How to run tensorboard from the container:  + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]  + # From the running container's command line (since it was run with 'bash' in the step above):  + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  # remove --bind_all for TF 1.15  + # Then open a browser:  + '''http://localhost:6006'''</font> Oleg

‎Notes

← Older revision Revision as of 21:56, 23 December 2019 (31 intermediate revisions by the same user not shown)Line 175: Line 175:    # Test 3: Run a local script (and include a local dir) in contatiner:   # Test 3: Run a local script (and include a local dir) in contatiner:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker  +  +  +==Walkthrough for CUDA 10.2 (Dec 2019)==  +  +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Will have to reboot  +  +===Docker===  +====Instructions====  +'''https://www.tensorflow.org/install/docker'''  +  +Quote:  + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).  +  +====Docker images====  +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/:  +{| class='wikitable'  +!TF version  +!Python major version  +!GPU support  +!TAG for Docker command  +|-  +|align='center'|1.15  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:1.15.0-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|3  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3'''  +|-  +|align='center'|2.0.0+  +|align='center'|2  +|align='center'|yes  +|<font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu'''  +|}  +  +====nvidia-docker====  +Somehow it was already installed.  +  +* Check NVIDIA docker version  + ~$ nvidia-docker version  +  +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 already register it.  +  +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  +  + ~$ '''docker run \'''  +    '''-it \'''  +    '''--rm \'''  +    '''--name devtest \'''  +    '''-p 0.0.0.0:6006:6006 \'''  +    '''--mount type=bind,source="$(pwd)"/target,target=/app \'''  +    '''--gpus all \'''  +    <font color='darkgreen'>'''tensorflow/tensorflow:latest-gpu-py3</font> \'''  +    '''bash'''</font>  +  +* How to run tensorboard from the container:  + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]  + # From the running container's command line (since it was run with 'bash' in the step above):  + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  + # Then open a browser:  + '''http://localhost:6006'''</font> Oleg

‎Notes

← Older revision Revision as of 21:20, 23 December 2019 (27 intermediate revisions by the same user not shown)Line 175: Line 175:    # Test 3: Run a local script (and include a local dir) in contatiner:   # Test 3: Run a local script (and include a local dir) in contatiner:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker  +  +  +==Walkthrough for CUDA 10.2 (Dec 2019)==  +  +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Will have to reboot  +  +===Docker===  +====Instructions====  +'''https://www.tensorflow.org/install/docker'''  +  +Quote:  + Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).  +  +====Docker images====  +Where to browse: https://hub.docker.com/r/tensorflow/tensorflow/  +* tag for python2 + gpu: '''tensorflow/tensorflow:latest-gpu'''  +* tag for python3 + gpu: '''tensorflow/tensorflow:latest-gpu-py3'''  +  +====nvidia-docker====  +Somehow it was already installed.  +  +* Check NVIDIA docker version  + ~$ nvidia-docker version  +  +* In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +* It's not immediately clear about the '''nvidia-container-runtime'''. nvidia-docker v1 & v2 already register it.  +  +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='2'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  +  + ~$ '''docker run \'''  +    '''-it \'''  +    '''--rm \'''  +    '''--name devtest \'''  +    '''-p 0.0.0.0:6006:6006 \'''  +    '''--mount type=bind,source="$(pwd)"/target,target=/app \'''  +    '''--gpus all \'''  +    '''tensorflow/tensorflow:latest-gpu-py3 \'''  +    '''bash'''</font>  +  +* How to run tensorboard from the container:  + <font size='2'># from [https://briancaffey.github.io/2017/11/20/using-tensorflow-and-tensor-board-with-docker.html here]  + # From the running container's command line (since it was run with 'bash' in the step above):  + root@e9efee9e3fd3:/# '''tensorboard --bind_all --logdir=/app/log.txt'''  + # Then open a browser:  + '''http://localhost:6006'''</font> Oleg

‎Walkthrough for CUDA 10.2 (Dec 2019)

← Older revision Revision as of 20:17, 23 December 2019 (10 intermediate revisions by the same user not shown)Line 175: Line 175:    # Test 3: Run a local script (and include a local dir) in contatiner:   # Test 3: Run a local script (and include a local dir) in contatiner:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker  +  +  +==Walkthrough for CUDA 10.2 (Dec 2019)==  +  +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Reboot  +  +===Docker===  +  +====Instructions====  +https://www.tensorflow.org/install/docker  +  +====nvidia-docker====  +* Check NVIDIA docker version  + ~$ nvidia-docker version  +  +In the docs it's clear that Docker version 19.03+ should use nvidia-docker2. For Docker of older versions - nvidia-docker v1 should be used.  +  +It's not immediately clear about the '''nvidia-container-runtime'''. What? Why? Is it automatically installed with nvidia-docker thing?  +  +====Notes====  +* Can mount a local directory in a 'binding' mode - i.e., update files locally so they are updated in the docker container as well:  + <font size='3'># this will bind-mount directory '''target''' located in '''$(pwd)''', which is a dir the command is run from  + # to '''/app''' in the docker container  + docker run \  +    --gpus all \  +    --name sometest \  +    --mount type=bind,source="$(pwd)"/target,target=/app \  +    -it \  +    tensorflow/tensorflow:latest-gpu \  +    bash</font> Oleg

‎Walkthrough for CUDA 10.2 (Dec 2019)

← Older revision Revision as of 19:15, 23 December 2019 (3 intermediate revisions by the same user not shown)Line 175: Line 175:    # Test 3: Run a local script (and include a local dir) in contatiner:   # Test 3: Run a local script (and include a local dir) in contatiner:    https://www.tensorflow.org/install/docker   https://www.tensorflow.org/install/docker  +  +  +==Walkthrough for CUDA 10.2 (Dec 2019)==  +  +===Install CUDA===  +* In this [https://www.tensorflow.org/install/gpu guide] there's a [https://developer.nvidia.com/cuda-toolkit-archive link to CUDA toolkit].  +** The toolkit (CUDA Toolkit 10.2) also updated the system driver to 440.33.01  +** Reboot Oleg

AndreyFilippov committed changes to the Elphel git project :
Started correction of the tiles distortions for lens distortions during correlation

← Older revision Revision as of 22:01, 5 December 2019 Line 9: Line 9:  |[https://arxiv.org/pdf/1911.06975.pdf Filippov, Andrey and Dzhimiev, Oleg "Long Range 3D with Quadocular Thermal (LWIR) Camera" arXiv preprint arXiv:1911.06975 (2019).] |[https://arxiv.org/pdf/1911.06975.pdf Filippov, Andrey and Dzhimiev, Oleg "Long Range 3D with Quadocular Thermal (LWIR) Camera" arXiv preprint arXiv:1911.06975 (2019).]  |Elphel publication |Elphel publication  +|-  +|2019  +|[https://www.researchgate.net/profile/Cui_Xiangbin/publication/335406867_The_conditions_of_the_formation_and_existence_of_Blue_Ice_Areas_in_the_ice_flow_transition_region_from_the_Antarctic_Ice_Sheet_to_the_Amery_Ice_Shelf_in_the_Larsemann_Hills_area/links/5d675ab5299bf11adf29bb92/The-conditions-of-the-formation-and-existence-of-Blue-Ice-Areas-in-the-ice-flow-transition-region-from-the-Antarctic-Ice-Sheet-to-the-Amery-Ice-Shelf-in-the-Larsemann-Hills-area.pdf Markov, Aleksey and Polyakov, Sergey and Sun, Bо and Lukin, Valeriy and Popov, Sergey and Yang, Huigen and Zhang, Tijun and Cui, Xiangbin and Guo, Jingxue and Cui, Penghui and others "Polar Science"]  +|Used Ephel camera in experimental setup  +  |- |-  |2019 |2019  |[https://documat.unirioja.es/descarga/articulo/6802185.pdf Campbell, Andrew, Alan Both, and Qian Chayn Sun. "Detecting and mapping traffic signs from Google Street View images using deep learning and GIS."] |[https://documat.unirioja.es/descarga/articulo/6802185.pdf Campbell, Andrew, Alan Both, and Qian Chayn Sun. "Detecting and mapping traffic signs from Google Street View images using deep learning and GIS."]  |Elphel cameras for GSV referenced   |Elphel cameras for GSV referenced    +  |- |-  |2019 |2019 Andrey.filippov

← Older revision Revision as of 01:12, 27 November 2019 (One intermediate revision by the same user not shown)Line 9: Line 9:  |[https://arxiv.org/pdf/1911.06975.pdf Filippov, Andrey and Dzhimiev, Oleg "Long Range 3D with Quadocular Thermal (LWIR) Camera" arXiv preprint arXiv:1911.06975 (2019).] |[https://arxiv.org/pdf/1911.06975.pdf Filippov, Andrey and Dzhimiev, Oleg "Long Range 3D with Quadocular Thermal (LWIR) Camera" arXiv preprint arXiv:1911.06975 (2019).]  |Elphel publication |Elphel publication  +|-  +|2019  +|[https://documat.unirioja.es/descarga/articulo/6802185.pdf Campbell, Andrew, Alan Both, and Qian Chayn Sun. "Detecting and mapping traffic signs from Google Street View images using deep learning and GIS."]  +|Elphel cameras for GSV referenced  +|-  +|2019  +|[https://documat.unirioja.es/descarga/articulo/6802185.pdf Díaz, Hernán Porras, Duvan Yahir Sanabria Echeverry, and Johan Alexander Ortiz Ferreira. "Tendencia mundial en tecnologías de sistemas de mapeo móvil implementadas con láser."]  +|Elphel Eyesis referenced  |- |-  |2019 |2019 Andrey.filippov

← Older revision Revision as of 21:34, 19 November 2019 (2 intermediate revisions by the same user not shown)Line 5: Line 5:  ! Citation/link ! Citation/link  ! Comments ! Comments  +|-  +|2019  +|[https://arxiv.org/pdf/1911.06975.pdf Filippov, Andrey and Dzhimiev, Oleg "Long Range 3D with Quadocular Thermal (LWIR) Camera" arXiv preprint arXiv:1911.06975 (2019).]  +|Elphel publication  |- |-  |2019 |2019 Line 19: Line 23:  |- |-  |2018 |2018 −|[https://arxiv.org/pdf/1811.08032 Filippov, Andrey, and Oleg Dzhimiev. "See far with TPNET: a Tile Processor and a CNN Symbiosis." arXiv preprint arXiv:1811.08032 (2018).]+|[https://arxiv.org/pdf/1811.08032 Filippov, Andrey, and Dzhimiev, Oleg. "See far with TPNET: a Tile Processor and a CNN Symbiosis." arXiv preprint arXiv:1811.08032 (2018).]  |Elphel publication |Elphel publication  |- |- Andrey.filippov

‎[SOLVED] Note 14: fixdep: Permission denied

← Older revision Revision as of 23:49, 3 October 2019 (6 intermediate revisions by the same user not shown)Line 260: Line 260:    ...   ...    ---[ end Kernel panic - not syncing: Fatal exception in interrupt   ---[ end Kernel panic - not syncing: Fatal exception in interrupt  +  +==<font color='green'>'''[SOLVED]'''</font> Note 14: fixdep: Permission denied==  +* Description:  + - We've had this error for a while, probably since kernel 4.0  + - usually happened when running do_compile_kernelmodules  + - EXTRA_OEMAKE = "-s -w '''-B''' KCFLAGS='-v'"  + - That '''-B''' forces to rebuild all targets and we also have '''-j8''' (in PARALLEL_MAKE variable) for the parallel build  +  - so when running the parallel build fixdep gets rebuilt several times and at some point  +    one of the targets (e.g. sortextable or kallsyms) calls it while fixdep is being compiled and overwritten for another target (probably)  +  - the exec rights are correct after the fact  +  +* Solution:  +  Removed '''-B'''. It make fixdep build only once and the problem is gone.  +  +* Note:  +  ~$ make -h  +    ...  +    -B, --always-make          Unconditionally make all targets.  +    ... Oleg

Oleg Dzhimiev committed changes to the Elphel git project :
updated - ran: scripts/used_file.py linux

Oleg Dzhimiev committed changes to the Elphel git project :
hide comments