Andrey Filippov (2bed5def) at 07 Apr 17:59 preparing for separate compilation in java3

Andrey Filippov (406eb708) at 07 Apr 17:08 preparing for separate compilation in java2 ... and 1 more commit

Andrey Filippov (39e75987) at 06 Apr 06:08 simplified by using larger output array

AndreyFilippov committed changes to the Elphel git project :
simplified by using larger output array

Andrey Filippov (0253bab4) at 06 Apr 06:07 simplified by using larger output array

Andrey Filippov (3f0b0bc0) at 06 Apr 05:20 fixed leak between colors in textures

AndreyFilippov committed changes to the Elphel git project :
fixed leak between colors in textures

Andrey Filippov (03329430) at 06 Apr 05:20 fixed leak between colors in textures

Andrey Filippov (10c327ae) at 05 Apr 21:13 debugging GPU OOB with large disparities, fixed in the kernel code

AndreyFilippov committed changes to the Elphel git project :
debugging GPU OOB with large disparities, fixed in the kernel code

AndreyFilippov committed changes to the Elphel git project :
debugging RGBA texture generation

AndreyFilippov committed changes to the Elphel git project :
updated to new GPU kernels

‎Note

← Older revision Revision as of 17:48, 1 April 2020 (3 intermediate revisions by the same user not shown)Line 28: Line 28:     ==<font color='blue'>Build</font>== ==<font color='blue'>Build</font>==  +===Note===  +* <font color='red'>While running bazel ate all RAM (have 16GB) a few times and PC "hanged". To limit bazel's appetites try:</font>  + <font size=2>~$ bazel build --jobs 4 --local_ram_resources=4096 ...  + ~$ bazel test --jobs 4 --local_ram_resources=4096 ...  + # I think that '''local_ram_resources''' is MBs per thread (have 8):  + ~$ bazel build --local_ram_resources=2048 ...  + ~$ bazel test --local_ram_resources=2048 ...</font>  +  ===Quick=== ===Quick===  ~/git/tensorflow-1.15.0/mvn_build.sh: ~/git/tensorflow-1.15.0/mvn_build.sh: Oleg

AndreyFilippov committed changes to the Elphel git project :
Using Marco's help to make jcuda work with nvrtc with cdp

AndreyFilippov committed changes to the Elphel git project :
debugging texture generation in GPU, fixed old NaN bug, similar to noticed in the GPU code

← Older revision Revision as of 23:36, 17 March 2020 Line 1: Line 1:  +{{Legacy}}  +'''Elphel currently uses git repository at https://git.elphel.com, mirrored at https://github.com/Elphel'''  +  +  = Releases description on Elphel's SourceForge project= = Releases description on Elphel's SourceForge project=    Andrey.filippov

‎Build

← Older revision Revision as of 23:43, 13 March 2020 (4 intermediate revisions by the same user not shown)Line 28: Line 28:     ==<font color='blue'>Build</font>== ==<font color='blue'>Build</font>==  +===Quick===  +~/git/tensorflow-1.15.0/mvn_build.sh:  + <font size=2># Step 1 (Java):  +  + bazel build -c opt //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni //tensorflow/java:pom  + mvn install:install-file -Dfile=bazel-bin/tensorflow/java/libtensorflow.jar -DpomFile=bazel-bin/tensorflow/java/pom.xml  +  + # Step 2 (JNI):  +  + bazel build -c opt //tensorflow/tools/lib_package:libtensorflow_jni.tar.gz  +  + rm -rf bazel-bin/tensorflow/tools/lib_package/maven  + mkdir -p bazel-bin/tensorflow/tools/lib_package/maven/org/tensorflow/native/linux-x86_64  +  + POM="  + <project>  +  <modelVersion>4.0.0</modelVersion>  +  <description>Platform-dependent native code for the TensorFlow Java library. CUDA support depends on the local build.</description>  +  <groupId>org.tensorflow</groupId>  +  <artifactId>libtensorflow_jni_gpu</artifactId>  +  <version>1.15.0</version>  +  <packaging>jar</packaging>  +  <build>  +    <resources>  +      <resource>  +        <directory>.</directory>  +          <excludes>  +            <exclude>target/**</exclude>  +          </excludes>  +      </resource>  +    </resources>  +  </build>  + </project>  + "  +  + echo $POM > bazel-bin/tensorflow/tools/lib_package/maven/pom.xml  +  + tar -zxvf bazel-bin/tensorflow/tools/lib_package/libtensorflow_jni.tar.gz -C bazel-bin/tensorflow/tools/lib_package/maven/org/tensorflow/native/linux-x86_64  + cd bazel-bin/tensorflow/tools/lib_package/maven  + mvn package  + mvn install  + cd ../../../../..  +  +===Detailed===  +    <font size=2>cd ~/git/tensorflow-1.15.0   <font size=2>cd ~/git/tensorflow-1.15.0    ./configure # do not forget CUDA   ./configure # do not forget CUDA Line 76: Line 121:  Note: '''libtensorflow_jni_gpu''' - name can be any - just make sure you use it in your project's pom.xml. Note: '''libtensorflow_jni_gpu''' - name can be any - just make sure you use it in your project's pom.xml.  Next: Next: −  <font size=2>mvn package+  <font size=2>cd bazel-bin/tensorflow/tools/lib_package/maven −  mvn install</font>+ mvn package  +  mvn install  + cd ../../../../..  +</font>     ==<font color='blue'>Install JAR to local Maven Repository</font>== ==<font color='blue'>Install JAR to local Maven Repository</font>== Oleg

‎Build

← Older revision Revision as of 23:43, 13 March 2020 (5 intermediate revisions by the same user not shown)Line 18: Line 18:  * '''Create Elipse project''' * '''Create Elipse project'''    −There's [https://github.com/bytedeco/javacpp-presets JavaCPP Presets] project. Seems useless.+There's [https://github.com/bytedeco/javacpp-presets JavaCPP Presets] project. <s>Seems useless</s>. Seems useful.     ==<font color='blue'>Install</font>== ==<font color='blue'>Install</font>== Line 28: Line 28:     ==<font color='blue'>Build</font>== ==<font color='blue'>Build</font>==  +===Quick===  +~/git/tensorflow-1.15.0/mvn_build.sh:  + <font size=2># Step 1 (Java):  +  + bazel build -c opt //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni //tensorflow/java:pom  + mvn install:install-file -Dfile=bazel-bin/tensorflow/java/libtensorflow.jar -DpomFile=bazel-bin/tensorflow/java/pom.xml  +  + # Step 2 (JNI):  +  + bazel build -c opt //tensorflow/tools/lib_package:libtensorflow_jni.tar.gz  +  + rm -rf bazel-bin/tensorflow/tools/lib_package/maven  + mkdir -p bazel-bin/tensorflow/tools/lib_package/maven/org/tensorflow/native/linux-x86_64  +  + POM="  + <project>  +  <modelVersion>4.0.0</modelVersion>  +  <description>Platform-dependent native code for the TensorFlow Java library. CUDA support depends on the local build.</description>  +  <groupId>org.tensorflow</groupId>  +  <artifactId>libtensorflow_jni_gpu</artifactId>  +  <version>1.15.0</version>  +  <packaging>jar</packaging>  +  <build>  +    <resources>  +      <resource>  +        <directory>.</directory>  +          <excludes>  +            <exclude>target/**</exclude>  +          </excludes>  +      </resource>  +    </resources>  +  </build>  + </project>  + "  +  + echo $POM > bazel-bin/tensorflow/tools/lib_package/maven/pom.xml  +  + tar -zxvf bazel-bin/tensorflow/tools/lib_package/libtensorflow_jni.tar.gz -C bazel-bin/tensorflow/tools/lib_package/maven/org/tensorflow/native/linux-x86_64  + cd bazel-bin/tensorflow/tools/lib_package/maven  + mvn package  + mvn install  + cd ../../../../..  +  +===Detailed===  +    <font size=2>cd ~/git/tensorflow-1.15.0   <font size=2>cd ~/git/tensorflow-1.15.0    ./configure # do not forget CUDA   ./configure # do not forget CUDA Line 76: Line 121:  Note: '''libtensorflow_jni_gpu''' - name can be any - just make sure you use it in your project's pom.xml. Note: '''libtensorflow_jni_gpu''' - name can be any - just make sure you use it in your project's pom.xml.  Next: Next: −  <font size=2>mvn package+  <font size=2>cd bazel-bin/tensorflow/tools/lib_package/maven −  mvn install</font>+ mvn package  +  mvn install  + cd ../../../../..  +</font>     ==<font color='blue'>Install JAR to local Maven Repository</font>== ==<font color='blue'>Install JAR to local Maven Repository</font>== Oleg

‎Build

← Older revision Revision as of 23:43, 13 March 2020 (19 intermediate revisions by the same user not shown)Line 18: Line 18:  * '''Create Elipse project''' * '''Create Elipse project'''    −There's [https://github.com/bytedeco/javacpp-presets JavaCPP Presets] project. Seems useless.+There's [https://github.com/bytedeco/javacpp-presets JavaCPP Presets] project. <s>Seems useless</s>. Seems useful.     ==<font color='blue'>Install</font>== ==<font color='blue'>Install</font>== Line 28: Line 28:     ==<font color='blue'>Build</font>== ==<font color='blue'>Build</font>==  +===Quick===  +~/git/tensorflow-1.15.0/mvn_build.sh:  + <font size=2># Step 1 (Java):  +  + bazel build -c opt //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni //tensorflow/java:pom  + mvn install:install-file -Dfile=bazel-bin/tensorflow/java/libtensorflow.jar -DpomFile=bazel-bin/tensorflow/java/pom.xml  +  + # Step 2 (JNI):  +  + bazel build -c opt //tensorflow/tools/lib_package:libtensorflow_jni.tar.gz  +  + rm -rf bazel-bin/tensorflow/tools/lib_package/maven  + mkdir -p bazel-bin/tensorflow/tools/lib_package/maven/org/tensorflow/native/linux-x86_64  +  + POM="  + <project>  +  <modelVersion>4.0.0</modelVersion>  +  <description>Platform-dependent native code for the TensorFlow Java library. CUDA support depends on the local build.</description>  +  <groupId>org.tensorflow</groupId>  +  <artifactId>libtensorflow_jni_gpu</artifactId>  +  <version>1.15.0</version>  +  <packaging>jar</packaging>  +  <build>  +    <resources>  +      <resource>  +        <directory>.</directory>  +          <excludes>  +            <exclude>target/**</exclude>  +          </excludes>  +      </resource>  +    </resources>  +  </build>  + </project>  + "  +  + echo $POM > bazel-bin/tensorflow/tools/lib_package/maven/pom.xml  +  + tar -zxvf bazel-bin/tensorflow/tools/lib_package/libtensorflow_jni.tar.gz -C bazel-bin/tensorflow/tools/lib_package/maven/org/tensorflow/native/linux-x86_64  + cd bazel-bin/tensorflow/tools/lib_package/maven  + mvn package  + mvn install  + cd ../../../../..  +  +===Detailed===  +    <font size=2>cd ~/git/tensorflow-1.15.0   <font size=2>cd ~/git/tensorflow-1.15.0    ./configure # do not forget CUDA   ./configure # do not forget CUDA Line 43: Line 88:     * '''xml''' and '''jar''' will be taken care of by '''mvn''' command. * '''xml''' and '''jar''' will be taken care of by '''mvn''' command. −* '''so''' will have to be in the library path. Link or copy to '''/usr/lib/''' or go with "java -Djava.library.path=...". Example:+* '''so''' will have to be in the library path (alternatively see [[Tensorflow_JNI_development#Build_so_package|'''Build so package''']] a little below and skip this linking). Link or copy to '''/usr/lib/''' or go with "java -Djava.library.path=...".  +   +===[option 1] Link so library===    <font size=2># /usr/lib is in the default java.library.path   <font size=2># /usr/lib is in the default java.library.path    sudo ln -sf ~/GIT/tensorflow-1.15.0/bazel-bin/tensorflow/java/libtensorflow_jni.so /usr/lib/</font>   sudo ln -sf ~/GIT/tensorflow-1.15.0/bazel-bin/tensorflow/java/libtensorflow_jni.so /usr/lib/</font>  +  +=== preferred [option 2] Build so package===  + <font size=2>bazel build -c opt //tensorflow/tools/lib_package:libtensorflow_jni.tar.gz</font>  +It puts all libs into a single archive.  +Now to create a JAR to replace [https://mvnrepository.com/artifact/org.tensorflow/libtensorflow_jni_gpu libtensorflow_jni_gpu], do this:  + <font size=2>mkdir -p bazel-bin/tensorflow/tools/lib_package/maven/org/tensorflow/native/linux-x86_64  + tar -zxvf bazel-bin/tensorflow/tools/lib_package/libtensorflow_jni.tar.gz -C bazel-bin/tensorflow/tools/lib_package/maven/org/tensorflow/native/linux-x86_64</font>  +Next create a pom.xml in ''bazel-bin/tensorflow/tools/lib_package/maven'':  + <font size=2><project>  +  <modelVersion>4.0.0</modelVersion>  +  <description>Platform-dependent native code for the TensorFlow Java library. CUDA support depends on the local build.</description>  +  <groupId>org.tensorflow</groupId>  +  <artifactId>'''libtensorflow_jni_gpu'''</artifactId>  +  <version>1.15.0</version>  +  <packaging>jar</packaging>  +  <build>  +    <resources>  +      <resource>  +        <directory>.</directory>  +          <excludes>  +            <exclude>target/**</exclude>  +          </excludes>  +      </resource>  +    </resources>  +  </build>  + </project></font>  +Note: '''libtensorflow_jni_gpu''' - name can be any - just make sure you use it in your project's pom.xml.  +Next:  + <font size=2>cd bazel-bin/tensorflow/tools/lib_package/maven  + mvn package  + mvn install  + cd ../../../../..  +</font>     ==<font color='blue'>Install JAR to local Maven Repository</font>== ==<font color='blue'>Install JAR to local Maven Repository</font>== Line 52: Line 132:  How to uninstall maven local repo - and switch back to official versions from Maven Central - [https://stackoverflow.com/questions/15358851/how-to-remove-jar-file-from-local-maven-repository-which-was-added-with-install this link]. Or remove unneeded stuff from '''~/.m2/repository/org/tensorflow''' How to uninstall maven local repo - and switch back to official versions from Maven Central - [https://stackoverflow.com/questions/15358851/how-to-remove-jar-file-from-local-maven-repository-which-was-added-with-install this link]. Or remove unneeded stuff from '''~/.m2/repository/org/tensorflow'''    −<font color=green size='4'>After '''*_jni.so''' is linked and '''jar''' installed one can resume normal development. See below what to add to your project's pom.xml</font>+<font color=green size='4'>After '''*_jni.so''' is linked (or '''jar''''d) and '''jar''' installed one can resume normal development. See below what to add to your project's pom.xml</font>     ==<font color='blue'>Modify TF JNI functions</font>== ==<font color='blue'>Modify TF JNI functions</font>== Oleg

← Older revision Revision as of 21:44, 13 March 2020 (18 intermediate revisions by the same user not shown)Line 18: Line 18:  * '''Create Elipse project''' * '''Create Elipse project'''    −There's [https://github.com/bytedeco/javacpp-presets JavaCPP Presets] project. Seems useless.+There's [https://github.com/bytedeco/javacpp-presets JavaCPP Presets] project. <s>Seems useless</s>. Seems useful.     ==<font color='blue'>Install</font>== ==<font color='blue'>Install</font>== Line 43: Line 43:     * '''xml''' and '''jar''' will be taken care of by '''mvn''' command. * '''xml''' and '''jar''' will be taken care of by '''mvn''' command. −* '''so''' will have to be in the library path. Link or copy to '''/usr/lib/''' or go with "java -Djava.library.path=...". Example:+* '''so''' will have to be in the library path (alternatively see [[Tensorflow_JNI_development#Build_so_package|'''Build so package''']] a little below and skip this linking). Link or copy to '''/usr/lib/''' or go with "java -Djava.library.path=...".  +   +===[option 1] Link so library===    <font size=2># /usr/lib is in the default java.library.path   <font size=2># /usr/lib is in the default java.library.path    sudo ln -sf ~/GIT/tensorflow-1.15.0/bazel-bin/tensorflow/java/libtensorflow_jni.so /usr/lib/</font>   sudo ln -sf ~/GIT/tensorflow-1.15.0/bazel-bin/tensorflow/java/libtensorflow_jni.so /usr/lib/</font>  +  +=== preferred [option 2] Build so package===  + <font size=2>bazel build -c opt //tensorflow/tools/lib_package:libtensorflow_jni.tar.gz</font>  +It puts all libs into a single archive.  +Now to create a JAR to replace [https://mvnrepository.com/artifact/org.tensorflow/libtensorflow_jni_gpu libtensorflow_jni_gpu], do this:  + <font size=2>mkdir -p bazel-bin/tensorflow/tools/lib_package/maven/org/tensorflow/native/linux-x86_64  + tar -zxvf bazel-bin/tensorflow/tools/lib_package/libtensorflow_jni.tar.gz -C bazel-bin/tensorflow/tools/lib_package/maven/org/tensorflow/native/linux-x86_64</font>  +Next create a pom.xml in ''bazel-bin/tensorflow/tools/lib_package/maven'':  + <font size=2><project>  +  <modelVersion>4.0.0</modelVersion>  +  <description>Platform-dependent native code for the TensorFlow Java library. CUDA support depends on the local build.</description>  +  <groupId>org.tensorflow</groupId>  +  <artifactId>'''libtensorflow_jni_gpu'''</artifactId>  +  <version>1.15.0</version>  +  <packaging>jar</packaging>  +  <build>  +    <resources>  +      <resource>  +        <directory>.</directory>  +          <excludes>  +            <exclude>target/**</exclude>  +          </excludes>  +      </resource>  +    </resources>  +  </build>  + </project></font>  +Note: '''libtensorflow_jni_gpu''' - name can be any - just make sure you use it in your project's pom.xml.  +Next:  + <font size=2>cd bazel-bin/tensorflow/tools/lib_package/maven  + mvn package  + mvn install  + cd ../../../../..  +</font>     ==<font color='blue'>Install JAR to local Maven Repository</font>== ==<font color='blue'>Install JAR to local Maven Repository</font>== Line 52: Line 87:  How to uninstall maven local repo - and switch back to official versions from Maven Central - [https://stackoverflow.com/questions/15358851/how-to-remove-jar-file-from-local-maven-repository-which-was-added-with-install this link]. Or remove unneeded stuff from '''~/.m2/repository/org/tensorflow''' How to uninstall maven local repo - and switch back to official versions from Maven Central - [https://stackoverflow.com/questions/15358851/how-to-remove-jar-file-from-local-maven-repository-which-was-added-with-install this link]. Or remove unneeded stuff from '''~/.m2/repository/org/tensorflow'''    −<font color=green size='4'>After '''*_jni.so''' is linked and '''jar''' installed one can resume normal development. See below what to add to your project's pom.xml</font>+<font color=green size='4'>After '''*_jni.so''' is linked (or '''jar''''d) and '''jar''' installed one can resume normal development. See below what to add to your project's pom.xml</font>     ==<font color='blue'>Modify TF JNI functions</font>== ==<font color='blue'>Modify TF JNI functions</font>== Oleg