Figure 1. Aircraft positions during descent captured with the quad stereo camera. Each animation frame corresponds to the available 3-D model.

While we continue to work on the multi-sensor stereo camera hardware (we plan to double the number of sensors to capture single-exposure HDR image sets) and develop code to get the ground truth data for the CNN training, we had some fun testing the camera for capturing aircraft position in 3-D space. Completely passive method, of course.

We found a suitable spot about 2.5 km from the beginning of the runway 34L of the Salt Lake City international airport (exact location is shown in the model viewer) so approaching aircraft would pass almost over our heads. With the 60°(H)×45°(V) field of view of the camera aircraft are visible when they are 270 m away horizontally.

Camera was set to capture synchronized image sets at 5 frames per second (same as in Fig. 1 animation), each set was processed independently without any relation to the other ones (no optical flow or motion vectors). Our existing (not airplane-aware) prototype software generated the scene 3-D models, they in turn were used to measure the aircraft positions. Result accuracy may be improved by using directly the intermediate Disparity Space Image (DSI) data – current reconstruction code tries to fit some flat surfaces to approximate the DSI data. At long distances there is insufficient data about the shape of the objects, so at ~500 m range (Fig. 2) the software used a fronto-parallel surface (distances to the aircraft nose, center, and tail are the same) and at ~600m it selected a tilted surface in the wrong direction, so the measured distance to the tail is larger than the one measured to the nose and center.

Figure 2. Ground distance and altitude of the three points sampled.

The measurements accuracy is still good, considering that the disparity of 1 image pixel corresponds for this short baseline camera to 527 meters range, at 800 m 1/10 pixel disparity precision corresponds to 120 m uncertainty. Ground speed and the descend rate (indicated in the animation) are calculated between the first captured set and the current one, so the last value averages over the full 6 seconds.

This method of capturing aircraft position and speed does not require that it travels directly to/from the camera, it can fly in any direction. There could also be multiple aircraft (such as UAVs) in the same camera view. Another 3-D scene (Fig. 3) captured from the same spot perpendicular to the flight direction shows an aircraft at 1600 m from the camera at 110 m altitude. According to the map the center line of the 34R runway is 14% farther (at 1820 m) from the camera position. That corresponds to 0.045 pix disparity error in the distance measurement, so calculated ground speed and altitude would also be 14% lower than actual.

Figure 3. Aircraft traveling perpendicular to the view direction.

Each of the provided 3-D scene models has the corresponding “set of 4 images” link on the settings page that can be viewed as an animation, manually iterated over or downloaded – the one for Fig. 3 scene shows how a 0.33 pixel wiggle of the FedEx MD-11 image looks like (images are zoomable and pannable). Here are the direct links for image sets of the first (1.75 pix disparity) and the last (0.7 pix disparity) of the scene models of the approaching B737 shown in Fig. 1.

These four images are not used in the actual reconstruction process, they are rendered from the source raw (JP4 format) ones for the specified disparity (D0.0 in the file names show that disparity was 0.0 pixels) for viewing or evaluating with alternative software. These images are not rectilinear and have residual common (for all four) radial distortion. They have optical aberrations corrected by the space-variant deconvolution, sub-camera residual misalignment is field-corrected (such misalignment may be caused by an uneven heating of the camera tubular structure by the direct sunlight). The images are partially rectified to the common radial distortion model that bests fits simultaneously all four sub-cameras. The remaining radial distortion is compensated during final generation of the 3-D model. These images have visible grid artifacts caused by the Bayer pattern – no de-mosaic processing is involved because it hurts linear operations over the images, and it is not adequate for the peripheral image areas (see earlier post). Each color component is processed separately for the phase correlation, when combining texture images these artifacts are reduced as they are not coherent between the channels. Any additional non-linear improvements of the Bayer mosaic suppression can be applied to the texture generation step, not to the data used for correlation.

AndreyFilippov committed changes to the Elphel git project :
disabled far objects detection when doing mismatch measurements

AndreyFilippov committed changes to the Elphel git project :
Tweaking fine adjustment

AndreyFilippov committed changes to the Elphel git project :
Updating batch mode of the 3d scene processing

AndreyFilippov committed changes to the Elphel git project :
cleaning up

AndreyFilippov committed changes to the Elphel git project :
Fixing field correction

AndreyFilippov committed changes to the Elphel git project :
Verified compatibility of the new code with other parts of the software

← Older revision Revision as of 00:17, 27 April 2018 (2 intermediate revisions by the same user not shown)Line 40: Line 40:    ~$ sudo pip3 install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.7.0-cp35-cp35m-linux_x86_64.whl   ~$ sudo pip3 install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.7.0-cp35-cp35m-linux_x86_64.whl    −* testing+* testing: unsupported card '''GeForce GT 610'''    ~$ python3   ~$ python3    Python 3.5.2 (default, Nov 23 2017, 16:37:01)     Python 3.5.2 (default, Nov 23 2017, 16:37:01)      [GCC 5.4.0 20160609] on linux   [GCC 5.4.0 20160609] on linux    Type "help", "copyright", "credits" or "license" for more information.   Type "help", "copyright", "credits" or "license" for more information. − >>>   − >>>   − >>>      >>> import tensorflow as tf   >>> import tensorflow as tf    >>> hello = tf.constant('Hello, World!')                                                                                                                                                                                >>> hello = tf.constant('Hello, World!')                                                                                                                                                                              Line 61: Line 58:    2018-04-26 13:00:19.181695: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0:  N                                                                                                                                                          2018-04-26 13:00:19.181695: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0:  N                                                                                                                                                           >>> print(sess.run(hello))                                                                                                                                                                                                                                >>> print(sess.run(hello))                                                                                                                                                                                                                               + b'Hello, World!'  +  +* testing: supported card '''GeForce GTX 750 Ti'''  + ~$ python3  + Python 3.5.2 (default, Nov 23 2017, 16:37:01)  + [GCC 5.4.0 20160609] on linux  + Type "help", "copyright", "credits" or "license" for more information.  + >>> import tensorflow as tf  + >>> hello = tf.constant('Hello, World!')   + >>> sess = tf.Session()  + 2018-04-26 18:14:05.427668: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning  + NUMA node zero  + 2018-04-26 18:14:05.428033: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:  + name: GeForce GTX 750 Ti major: 5 minor: 0 memoryClockRate(GHz): 1.1105  + pciBusID: 0000:01:00.0  + totalMemory: 1.95GiB freeMemory: 1.53GiB  + 2018-04-26 18:14:05.428061: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0  + 2018-04-26 18:14:05.927106: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:  + 2018-04-26 18:14:05.927149: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917]      0  + 2018-04-26 18:14:05.927163: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0:  N  + 2018-04-26 18:14:05.927313: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1289 MB memory) -> physical GPU (device: 0,  + name: GeForce GTX 750 Ti, pci bus id: 0000:01:00.0, compute capability: 5.0)  + >>> print(sess.run(hello))    b'Hello, World!'   b'Hello, World!'    Oleg

← Older revision Revision as of 00:17, 27 April 2018 (11 intermediate revisions by the same user not shown)Line 5: Line 5:    81:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)   81:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)  * check driver version * check driver version −  ~$ modinfo nv[TAB]+  ~$ cat /proc/driver/nvidia/version  + NVRM version: NVIDIA UNIX x86_64 Kernel Module  387.26  Thu Nov  2 21:20:16 PDT 2017  + GCC version:  gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)  * install cuda 9.0 with patches * install cuda 9.0 with patches    https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal   https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal Line 15: Line 17:    ~$ export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}   ~$ export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}    ~$ export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}   ~$ export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}  +*  + ~$ nvidia-smi  + Thu Apr 26 12:39:25 2018       + +-----------------------------------------------------------------------------+  + | NVIDIA-SMI 387.26                Driver Version: 387.26                    |  + |-------------------------------+----------------------+----------------------+  + | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |  + | Fan  Temp  Perf  Pwr:Usage/Cap|        Memory-Usage | GPU-Util  Compute M. |  + |===============================+======================+======================|  + |  0  GeForce GT 610      Off  | 00000000:81:00.0 N/A |                  N/A |  + | N/A  31C    P8    N/A /  N/A |    148MiB /  956MiB |    N/A      Default |  + +-------------------------------+----------------------+----------------------+  +                                                                                 + +-----------------------------------------------------------------------------+  + | Processes:                                                      GPU Memory |  + |  GPU      PID  Type  Process name                            Usage      |  + |=============================================================================|  + |    0                    Not Supported                                      |  + +-----------------------------------------------------------------------------+  +  +* install tensorflow  + ~$ sudo pip3 install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.7.0-cp35-cp35m-linux_x86_64.whl  +  +* testing: unsupported card '''GeForce GT 610'''  + ~$ python3  + Python 3.5.2 (default, Nov 23 2017, 16:37:01)  + [GCC 5.4.0 20160609] on linux  + Type "help", "copyright", "credits" or "license" for more information.  + >>> import tensorflow as tf  + >>> hello = tf.constant('Hello, World!')                                                                                                                                                                             + >>> sess = tf.Session()                                                                                                                                                                                                 + 2018-04-26 13:00:19.050625: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA                                           + 2018-04-26 13:00:19.181581: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:                                                                                                             + name: GeForce GT 610 major: 2 minor: 1 memoryClockRate(GHz): 1.62                                                                                                                                                                 + pciBusID: 0000:81:00.0                                                                                                                                                                                                           + totalMemory: 956.50MiB freeMemory: 631.69MiB                                                                                                                                                                                               + <font color='red'><b>2018-04-26 13:00:19.181648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1394] Ignoring visible gpu device (device: 0, name: GeForce GT 610, pci bus id: 0000:81:00.0, compute capability: 2.1) with Cuda compute capability 2.1. The minimum required Cuda capability is 3.5.</b></font>                                                                                                                                                                                                               + 2018-04-26 13:00:19.181669: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:                                                                                       + 2018-04-26 13:00:19.181683: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917]      0                                                                                                                                                       + 2018-04-26 13:00:19.181695: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0:  N                                                                                                                                                       + >>> print(sess.run(hello))                                                                                                                                                                                                                             + b'Hello, World!'  +  +* testing: supported card '''GeForce GTX 750 Ti'''  + ~$ python3  + Python 3.5.2 (default, Nov 23 2017, 16:37:01)  + [GCC 5.4.0 20160609] on linux  + Type "help", "copyright", "credits" or "license" for more information.  + >>> import tensorflow as tf  + >>> hello = tf.constant('Hello, World!')   + >>> sess = tf.Session()  + 2018-04-26 18:14:05.427668: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning  + NUMA node zero  + 2018-04-26 18:14:05.428033: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:  + name: GeForce GTX 750 Ti major: 5 minor: 0 memoryClockRate(GHz): 1.1105  + pciBusID: 0000:01:00.0  + totalMemory: 1.95GiB freeMemory: 1.53GiB  + 2018-04-26 18:14:05.428061: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0  + 2018-04-26 18:14:05.927106: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:  + 2018-04-26 18:14:05.927149: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917]      0  + 2018-04-26 18:14:05.927163: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0:  N  + 2018-04-26 18:14:05.927313: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1289 MB memory) -> physical GPU (device: 0,  + name: GeForce GTX 750 Ti, pci bus id: 0000:01:00.0, compute capability: 5.0)  + >>> print(sess.run(hello))  + b'Hello, World!'  +  +* Supported cards:  + https://developer.nvidia.com/cuda-gpus  +  +* As a quickfix had to install CuDNN 7.0.5 instead of latest:  + https://stackoverflow.com/questions/49960132/cudnn-library-compatibility-error-after-loading-model-weights Oleg

← Older revision Revision as of 20:53, 26 April 2018 (8 intermediate revisions by the same user not shown)Line 5: Line 5:    81:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)   81:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)  * check driver version * check driver version −  ~$ modinfo nv[TAB]+  ~$ cat /proc/driver/nvidia/version  + NVRM version: NVIDIA UNIX x86_64 Kernel Module  387.26  Thu Nov  2 21:20:16 PDT 2017  + GCC version:  gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)  * install cuda 9.0 with patches * install cuda 9.0 with patches    https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal   https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal Line 15: Line 17:    ~$ export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}   ~$ export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}    ~$ export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}   ~$ export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}  +*  + ~$ nvidia-smi  + Thu Apr 26 12:39:25 2018       + +-----------------------------------------------------------------------------+  + | NVIDIA-SMI 387.26                Driver Version: 387.26                    |  + |-------------------------------+----------------------+----------------------+  + | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |  + | Fan  Temp  Perf  Pwr:Usage/Cap|        Memory-Usage | GPU-Util  Compute M. |  + |===============================+======================+======================|  + |  0  GeForce GT 610      Off  | 00000000:81:00.0 N/A |                  N/A |  + | N/A  31C    P8    N/A /  N/A |    148MiB /  956MiB |    N/A      Default |  + +-------------------------------+----------------------+----------------------+  +                                                                                 + +-----------------------------------------------------------------------------+  + | Processes:                                                      GPU Memory |  + |  GPU      PID  Type  Process name                            Usage      |  + |=============================================================================|  + |    0                    Not Supported                                      |  + +-----------------------------------------------------------------------------+  +  +* install tensorflow  + ~$ sudo pip3 install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.7.0-cp35-cp35m-linux_x86_64.whl  +  +* testing  + ~$ python3  + Python 3.5.2 (default, Nov 23 2017, 16:37:01)  + [GCC 5.4.0 20160609] on linux  + Type "help", "copyright", "credits" or "license" for more information.  + >>>  + >>>  + >>>  + >>> import tensorflow as tf  + >>> hello = tf.constant('Hello, World!')                                                                                                                                                                             + >>> sess = tf.Session()                                                                                                                                                                                                 + 2018-04-26 13:00:19.050625: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA                                           + 2018-04-26 13:00:19.181581: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:                                                                                                             + name: GeForce GT 610 major: 2 minor: 1 memoryClockRate(GHz): 1.62                                                                                                                                                                 + pciBusID: 0000:81:00.0                                                                                                                                                                                                           + totalMemory: 956.50MiB freeMemory: 631.69MiB                                                                                                                                                                                               + <font color='red'><b>2018-04-26 13:00:19.181648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1394] Ignoring visible gpu device (device: 0, name: GeForce GT 610, pci bus id: 0000:81:00.0, compute capability: 2.1) with Cuda compute capability 2.1. The minimum required Cuda capability is 3.5.</b></font>                                                                                                                                                                                                               + 2018-04-26 13:00:19.181669: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:                                                                                       + 2018-04-26 13:00:19.181683: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917]      0                                                                                                                                                       + 2018-04-26 13:00:19.181695: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0:  N                                                                                                                                                       + >>> print(sess.run(hello))                                                                                                                                                                                                                             + b'Hello, World!'  +  +* Supported cards:  + https://developer.nvidia.com/cuda-gpus  +  +* As a quickfix had to install CuDNN 7.0.5 instead of latest:  + https://stackoverflow.com/questions/49960132/cudnn-library-compatibility-error-after-loading-model-weights Oleg

← Older revision Revision as of 19:49, 26 April 2018 (7 intermediate revisions by the same user not shown)Line 5: Line 5:    81:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)   81:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)  * check driver version * check driver version −  ~$ modinfo nv[TAB]+  ~$ cat /proc/driver/nvidia/version  + NVRM version: NVIDIA UNIX x86_64 Kernel Module  387.26  Thu Nov  2 21:20:16 PDT 2017  + GCC version:  gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)  * install cuda 9.0 with patches * install cuda 9.0 with patches    https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal   https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal Line 15: Line 17:    ~$ export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}   ~$ export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}    ~$ export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}   ~$ export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}  +*  + ~$ nvidia-smi  + Thu Apr 26 12:39:25 2018       + +-----------------------------------------------------------------------------+  + | NVIDIA-SMI 387.26                Driver Version: 387.26                    |  + |-------------------------------+----------------------+----------------------+  + | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |  + | Fan  Temp  Perf  Pwr:Usage/Cap|        Memory-Usage | GPU-Util  Compute M. |  + |===============================+======================+======================|  + |  0  GeForce GT 610      Off  | 00000000:81:00.0 N/A |                  N/A |  + | N/A  31C    P8    N/A /  N/A |    148MiB /  956MiB |    N/A      Default |  + +-------------------------------+----------------------+----------------------+  +                                                                                 + +-----------------------------------------------------------------------------+  + | Processes:                                                      GPU Memory |  + |  GPU      PID  Type  Process name                            Usage      |  + |=============================================================================|  + |    0                    Not Supported                                      |  + +-----------------------------------------------------------------------------+  +  +* install tensorflow  + ~$ sudo pip3 install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.7.0-cp35-cp35m-linux_x86_64.whl  +  +* testing  + ~$ python3  + Python 3.5.2 (default, Nov 23 2017, 16:37:01)  + [GCC 5.4.0 20160609] on linux  + Type "help", "copyright", "credits" or "license" for more information.  + >>>  + >>>  + >>>  + >>> import tensorflow as tf  + >>> hello = tf.constant('Hello, World!')                                                                                                                                                                             + >>> sess = tf.Session()                                                                                                                                                                                                 + 2018-04-26 13:00:19.050625: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA                                           + 2018-04-26 13:00:19.181581: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:                                                                                                             + name: GeForce GT 610 major: 2 minor: 1 memoryClockRate(GHz): 1.62                                                                                                                                                                 + pciBusID: 0000:81:00.0                                                                                                                                                                                                           + totalMemory: 956.50MiB freeMemory: 631.69MiB                                                                                                                                                                                               + <font color='red'><b>2018-04-26 13:00:19.181648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1394] Ignoring visible gpu device (device: 0, name: GeForce GT 610, pci bus id: 0000:81:00.0, compute capability: 2.1) with Cuda compute capability 2.1. The minimum required Cuda capability is 3.5.</b></font>                                                                                                                                                                                                               + 2018-04-26 13:00:19.181669: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:                                                                                       + 2018-04-26 13:00:19.181683: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917]      0                                                                                                                                                       + 2018-04-26 13:00:19.181695: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0:  N                                                                                                                                                       + >>> print(sess.run(hello))                                                                                                                                                                                                                             + b'Hello, World!'  +  +* Supported cards:  + https://developer.nvidia.com/cuda-gpus Oleg

← Older revision Revision as of 18:42, 26 April 2018 (3 intermediate revisions by the same user not shown)Line 5: Line 5:    81:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)   81:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)  * check driver version * check driver version −  ~$ modinfo nv[TAB]+  ~$ cat /proc/driver/nvidia/version  + NVRM version: NVIDIA UNIX x86_64 Kernel Module  387.26  Thu Nov  2 21:20:16 PDT 2017  + GCC version:  gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)  * install cuda 9.0 with patches * install cuda 9.0 with patches    https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal   https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal Line 15: Line 17:    ~$ export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}   ~$ export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}    ~$ export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}   ~$ export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}  +*  + ~$ nvidia-smi  + Thu Apr 26 12:39:25 2018       + +-----------------------------------------------------------------------------+  + | NVIDIA-SMI 387.26                Driver Version: 387.26                    |  + |-------------------------------+----------------------+----------------------+  + | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |  + | Fan  Temp  Perf  Pwr:Usage/Cap|        Memory-Usage | GPU-Util  Compute M. |  + |===============================+======================+======================|  + |  0  GeForce GT 610      Off  | 00000000:81:00.0 N/A |                  N/A |  + | N/A  31C    P8    N/A /  N/A |    148MiB /  956MiB |    N/A      Default |  + +-------------------------------+----------------------+----------------------+  +                                                                                 + +-----------------------------------------------------------------------------+  + | Processes:                                                      GPU Memory |  + |  GPU      PID  Type  Process name                            Usage      |  + |=============================================================================|  + |    0                    Not Supported                                      |  + +-----------------------------------------------------------------------------+ Oleg

Created page with "==Pre== * check device ~$ lspci | grep NVIDIA 81:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 610] (rev a1) 81:00.1 Audio device: NVIDIA Corporation..."

New page

==Pre==
* check device
~$ lspci | grep NVIDIA
81:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 610] (rev a1)
81:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)
* check driver version
~$ modinfo nv[TAB]
* install cuda 9.0 with patches
https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal
* then
sudo apt-get install cuda-9-0

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions
*
~$ export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}
~$ export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} Oleg

Oleg Dzhimiev committed changes to the Elphel git project :
added cron

Oleg Dzhimiev committed changes to the Elphel git project :
read rtc even if OF flag is set, report

Oleg Dzhimiev committed changes to the Elphel git project :
auto exposure on/off

AndreyFilippov committed changes to the Elphel git project :
Debugging LMA for correlation maximums

AndreyFilippov committed changes to the Elphel git project :
fitting correlation maximum with LMA

Fig.1 MT9F002

MT9F002

This post briefly covers implementation of a driver for On Semi’s MT9F002 14MPx image sensor for 10393 system boards – the steps are more or less general. The driver is included in the latest software/firmware image (20180416). The implemented features are programmable:

• window size
• horizontal & vertical mirror
• color gains
• exposure
• fps and trigger-synced ports
• frame-based commands sequence allowing to change settings of any image up to 16 frames ahead (didn’t need to be implemented as it’s the common part of the driver for all sensors)
• auto cable phase adjustment during init for cables of various lengths

 

Setup

• Get the register reference document for MT9F002
• Setup the Eclipse IDE for linux kernel development – see instructions

 

Capturing images without the driver

There is a set of scripts used for verification in software (on PC) and hardware (on the target board). Among other things they can be used to establish initial communication with a sensor, run registers initialization and acquire the first images. The process for MT9F002 is described on our wiki.
 

Driver implementation

In general the driver can be split into 2 parts:

• common for all sensors – sets up working with fpga frame-based command sequencers (see NC393 development progress and the future plans).
• sensor specific – commands that do actual communication with a sensor. This is the part needs to be implemented.

For a sensor there are a few tables and a few functions, the most important ones are:

tables:

• mt9f002_pages – a ‘page’ is the upper byte of a 16-bit register address of the sensor (for 8-bit reg addresses the page number is 0x00), the table should list all available pages in the sensor – or at least the most important ones – they are used to program fpga’s 256-max records per port tables used to write commands and at least one record is used to read from port – so in general for a setup w/o a MUX board the limit of allowed pages per sensor is 256-1 = 255 (with a MUX + 3x sensors on a single port it is 256-2(mux write & read)-4(3x sensor read + 1x broadcast read))/4 = 62)
• mt9f002_par2addr – used convert 256 8-bit parameters available as P_SENSOR_REGSxxx to 16-bit register address
• mt9f002_ahead_tab – sequencer commands latencies – leave default, they are needed on the final stages of development

functions (they are registered on top of the common ones):

• mt9f002_pgm_detectsensor – remove reset read sensor id
• mt9f002_pgm_initsensor – init registers that select interface, sensor’s PLL multipliers/dividers and other required settings
• mt9f002_pgm_exposure – program exposure
• mt9f002_pgm_window – program image width and height
• mt9f002_pgm_limitfps – limit fps based on compressor capabilities and trigger period settings
• mt9f002_pgm_gains – program color gains
• mt9f002_pgm_triggermode – set sensor in triggered mode
• mt9f002_pgm_sensorregs – write sensor registers directly

See this article for more details.

 

‎Changelog

← Older revision Revision as of 23:55, 19 April 2018 Line 287: Line 287:  ===Changelog=== ===Changelog===    <font size='1'>'''==20180416=='''   <font size='1'>'''==20180416=='''  + * 20180419 - mt9f002: fixed changing window size    * mt9f002: added triggered mode   * mt9f002: added triggered mode    * mt9f002: added vertical and horizontal flips (mirror)   * mt9f002: added vertical and horizontal flips (mirror) Oleg