Oleg Dzhimiev committed changes to the Elphel git project :
moved patch to tree

Oleg Dzhimiev committed changes to the Elphel git project :
updated

Elphel uses embedded GNU/Linux distribution based on Yocto. For most of our development (excluding just mechanical and PCB design) we use universal Eclipse IDE: for FPGA development, Linux kernel drivers development, embedded applications and web applications, for editing LaTeX texts. And we use this popular IDE for delivering pre-configured projects to our users to make it easier for them to start efficient modification of the initial camera software and then initiate the new projects.

We tried to use Yocto plugin but were not able to configure it for kernel development, and the kernel drivers development is one of the the largest and probably is the most difficult part of the camera software development, the part were we need Eclipse IDE assistance most.

One of the major challenges of using code analysis tools of Eclipse CDT with the Linux kernel is that there are so many files that define the same names. These files are selected during the build process, and for correct code analysis Eclipse CDT has to reproduce rather complex Linux system of configuration, multi-level macro defines to resolve references in the source code. We were able to solve this problem (to some extent), but it required a fair amount of manual tweaking and was not universal – developing applications would require different modifications.

Figure 1. Excluded source directories in the Navigator (left) panel are crossed out.

At the same time all the software components in the distribution are built with the powerful Bitbake build system, and existing ”recipes” and the invoked Makefiles “know” which files to use. Following DRY (“don’t repeat yourself”) principle we removed references to “make” command in the project build and replaced them with bitbake command “bitbake <target> -c compile -f” running in the Eclipse console. As we did that for the main build command (CDT Builder) the console output is parsed for errors and warnings, results appear as problem markers in “Problems” view and in the source code. To help Eclipse CDT (and users who navigate the source code with it) limit attention to only files and directories that are actually used in the bitbake build process we implemented the following trick (and coded it as used_files.py script):

• Initialize source/headers directories with bitbake, so it “knows” that everything needs to be rebuilt for the project
• Create a list of the source files (resolving symlinks when needed) and “touch” them, setting modification timestamps. This action prepares the files so the next (first after modification) file access will be recorded as access timestamp. Record the current time.
• Wait a few seconds to reliably distinguish if each file was accessed after modification
• Run bitbake build (“bitbake <target> -c compile -f”)
• Scan all the files from the previously created source list and generate ”include_list” of those that were accessed during the build process.
• As CDT accepts only “exclude” filters in this context, recursively combine full source list and include_list to generate “exclude_list” pruning all the branches that have nothing to include and replacing them with the full branch reference
• Apply the generated exclusion list to the CDT project file “.cproject”

In the case of such complex system as Linux kernel there still remain several incorrectly resolved links, because different parts of the code may use different header files that share the same names, but there are not many of them, they can be handled manually.

Elphel wiki page Eclipse_CDT_projects_with_bitbake has more details on using Elphel camera projects with Eclipse IDE.

Oleg Dzhimiev committed changes to the Elphel git project :
updated

Oleg Dzhimiev committed changes to the Elphel git project :
uartps

Oleg Dzhimiev committed changes to the Elphel git project :
update micron driver

Oleg Dzhimiev committed changes to the Elphel git project :
udpated

Oleg Dzhimiev committed changes to the Elphel git project :
updated

Oleg Dzhimiev committed changes to the Elphel git project :
updated

Oleg Dzhimiev committed changes to the Elphel git project :
updated

Oleg Dzhimiev committed changes to the Elphel git project :
updated

Oleg Dzhimiev committed changes to the Elphel git project :
+comment

Oleg Dzhimiev committed changes to the Elphel git project :
updated

Oleg Dzhimiev committed changes to the Elphel git project :
updated

Oleg Dzhimiev committed changes to the Elphel git project :
changes to nand_base

‎Setting up New Project

Show changes Andrey.filippov

‎Setting up New Project

Show changes Andrey.filippov

Show changes Andrey.filippov

‎Initializing Eclipse Workspace for Existing (Cloned) Projects

← Older revision Revision as of 03:16, 22 November 2017 (17 intermediate revisions by the same user not shown)Line 1: Line 1:  +==Introduction==  +Elphel camera software is based on Yocto Poky distribution, all the individual projects such as Linux kernel drivers, C/C++ applications, web applications, Python projects are organized as '''Makefile''' projects, each tied to a separate Git repository. The makefiles are not standalone, they receive defined variables provided by '''bitbake''' recipes and classes.     +The top project [https://git.elphel.com/Elphel/elphel393 elphel393] has the '''setup.py''' script that installs and synchronizes them. All projects have Eclipse IDE configuration data (in '''eclipse_project_setup''' subdirectories), and this data is copied to the project root by '''setup.py'''. As Eclipse project settings may have user-dependent settings, current configuration files (/.project, /.cproject, ...) are not version controlled, so to restore them after corruption or just to get updated versions you need either to manually copy '''eclipse_project_setup''' sub-tree, or delete /.project and re-run '''setup.py'''.    − +Eclipse IDE setting reference not only files that are in the Git repository, but also those that are generated during build process, so Eclipse should be launched with these projects only after the command-line [[Poky_2.0_manual#Setup|build process]] has finished. If Eclipse is opened after some project is cleared (such as with '''bitbake -c clean''') it will delete the non-existing directories from '''.cproject''' configuration files. If that happens, you may close Eclipse and restore '''.cproject''' from the '''eclipse_project_setup/.cproject'''. −  ├── bitbake-logs -> /home/eyesis/nc393/elphel393/poky/build/tmp/work/cortexa9-neon-poky-linux-gnueabi/apps-gps/1_0-26/temp+  −  ├── eclipse_project_setup+   +After running '''setup.py''' for the first time you may create the brand [[#Initializing_Eclipse_Workspace_for_Existing_.28Cloned.29_Projects|new workspace]] for Eclipse.  +Projects are relying on '''bitbake''' build process, so main Eclipse build command launches '''bitbake ''target-name'' -c compile -f''' instead of '''make all''' or similar. And as error processing (problem highlighting in the source code, jump to source file/line) is possible only for the main builder, it is the '''CDT Builder''' that has to be configured this way.  +   +   +==Project Files and Directories==  +   +Below is the file/directory list of a typical application ('''elphel-apps-astreamer''' is a subdirectory of '''elphel393/rootfs-elphel''', '''apps-astreamer''' - bitbake target for this project):  +   +  ├── '''bitbake-logs''' -> /home/eyesis/nc393/elphel393/poky/build/tmp/work/cortexa9-neon-poky-linux-gnueabi/apps-astreamer/1_0-23/temp  +  ├── '''eclipse_project_setup'''    │   ├── .externalToolBuilders   │   ├── .externalToolBuilders −  │   │   └── SCP apps-gps to target.launch+  │   │   └── SCP apps-astreamer to target.launch    │   ├── .cproject   │   ├── .cproject    │   └── .project   │   └── .project −  ├── .externalToolBuilders+  ├── '''.externalToolBuilders''' −  │   └── SCP apps-gps to target.launch+  │   └── SCP apps-astreamer to target.launch −  ├── image -> /home/eyesis/nc393/elphel393/poky/build/tmp/work/cortexa9-neon-poky-linux-gnueabi/apps-gps/1_0-26/image+  ├── '''.git''' −  ├── scripts -> ../../scripts/+  ... −  │   ├── run_bitbake.sh+ ├── '''image''' -> /home/eyesis/nc393/elphel393/poky/build/tmp/work/cortexa9-neon-poky-linux-gnueabi/apps-astreamer/1_0-23/image −  │   └── used_files.py+  ├── '''scripts''' -> ../../scripts/ −  ├── src+  │  ├── '''run_bitbake.sh''' − │   ├── .depend+  │  └── '''used_files.py''' −  │   ├── exifgen.c+  ├── '''.settings''' − │   ├── exifgen.h+  │   └── language.settings.xml −  │   ├── garmin.h+  ├── '''src''' −  │   ├── garminusb2exif.c+  │   ├── audio.cpp −  │   ├── garminusb2nmea.c+  │   ├── audio.h − │   ├── imu_setup.php+  ... − │   ├── logger_launcher.php+  │   ├── streamer.h − │   ├── log_imu.c+  │   ├── video.cpp −  │   ├── Makefile+  │   └── video.h − │   ├── nmea2exif+  ├── '''sysroots''' -> /home/eyesis/nc393/elphel393/poky/build/tmp/sysroots − │   ├── nmea2exif.c+  ├── '''.cproject''' − │   ├── nmeagen.c+  ├── '''.gitignore''' − │   ├── nmeagen.h+  ├── '''LICENSE''' −  │   ├── read_imu_log.php+  ├── '''Makefile''' −  │   └── start_gps_compass.php+  ├── '''.project''' −  ├── sysroots -> /home/eyesis/nc393/elphel393/poky/build/tmp/sysroots+  ├── '''README.md''' −  ├── .cproject+  └── '''VERSION''' −  ├── .gitignore+*'''bitbake-logs''' is a convenience link to the latest bitbake build logs −  ├── LICENSE+*'''eclipse_project_setup''' copy of Eclipse IDE  settings under version control. Content of the subdirectory is copied (during setup or manually) to teh project root directory (one level up). −  ├── Makefile+*'''.externalToolBuilders''' is copied from '''eclipse_project_setup''' and then maintained by Eclipse. Currently is used to launch file copying to the target (actual camera). −  ├── .project+*'''.git''' - project Git data tree −  ├── README.md+*'''image''' is a link to directory that contains deployable files, built as a result of bitbake recipes execution. −  └── VERSION+*'''scripts''' - a link to a collection of scripts needed for all projects. It contains the following files:  +**'''run_bitbake.sh''' - script to create an environment and execute bitbake recipe. Used every time Eclipse builds the project.  +**'''used_files.py''' - script to create a list of header files/directories used by bitbake to build the project. It is not needed to run to configure existing projects - all of them are provided with preconfigured '''.cproject''' files. ''TODO: add link to description''  +*'''.settings''' - project settings maintained by Eclipse IDE  +*'''src''' - a hard-coded (in '''used_files.py''') directory name for C/C++ source files. Source files should not be in the project root directory for convenience of filter generation for additional source (headers) directories  +*'''sysroots''' - a link to header files directories, available in the target system. The content is generated after the linux kernel is being built. All the applications use header files from the subdirectories of this folder. This link is generated for every project by the bitbake class.  +*'''.cproject''' - Eclipse CDT project configuration. Parts of this file are modified by the scripts, such as '''used_files.py'''. This file is copied from version-controlled '''eclipse_project_setup''' during setup or manually.  +*'''.gitignore''' - Git ignore file list  +*'''LICENSE''' - project GNU GPLv3 license  +*'''Makefile''' - top make file of the project. It is invoked when executing bitbake recipes, it receives variables defined in bitbake recipes/classes  +*'''.project''' - Eclipse general project configuration. This file is copied from version-controlled '''eclipse_project_setup''' during setup or manually.  +*'''README.md''' - README file presented by the Git repository web interface  +*'''VERSION''' - project version, used by bitbake build process to determine which dependent recipes have to be re-ran.  +==Initializing Eclipse Workspace for Existing (Cloned) Projects==  +After running '''setup.py''' for the first time you may create the brand new workspace for Eclipse, that should have all necessary plugins (such as CDT and EGit at least) already installed. Workspace is created with '''setup_eclipse.py''' script, list of projects that it sets up is saved in '''setup_eclipse_paths.xml''' file. '''setup_eclipse.py''' provides usage information when launched without parameters, for normal run it requires full path to Eclipse installation - directory that contains '''eclipse''' executable and '''eclipse.ini''' configuration. Before running '''setup_eclipse.py''' Eclipse should have all the required plugins installed (at least CDT and EGit) using temporary workspace.  +   +'''setup_eclipse.py''' performs 2 actions:  +*It first batch-imports all the listed in '''setup_eclipse_paths.xml''' projects using headless build of the CDT (org.eclipse.cdt.managedbuilder.core.headlessbuild), without actually building and indexing files - indexing should be done manually before using particular project.  +*Next it connects project Git repositories to Eclipse EGit, so the projects are immediately ready for Git operations ('team' operations). project navigator shows repository and branch for each of the projects.     +After creating a new workspace with '''setup_eclipse.py''' it can be opened with Eclipse, the projects you plan to use have to be indexed (right-click on the project in the Navigator ⇒ Index ⇒ Rebuild.     +==a==  +==b==  +==c==  [[File:Eclipse builders.png|500px|thumb|right|Figure 1. Project builders]] [[File:Eclipse builders.png|500px|thumb|right|Figure 1. Project builders]]  [[File:Eclipse scp main.png|500px|thumb|right|Figure 2. Configuring SCP to target tool for apps-astreamer project]] [[File:Eclipse scp main.png|500px|thumb|right|Figure 2. Configuring SCP to target tool for apps-astreamer project]]  [[File:Eclipse scp options.png|500px|thumb|right|Figure 3. Configuring SCP to target tool for apps-astreamer project(options)]] [[File:Eclipse scp options.png|500px|thumb|right|Figure 3. Configuring SCP to target tool for apps-astreamer project(options)]]  [[File:Eclipse ccpp build.png|500px|thumb|right|Figure 4. C/C++ Build ⇒ Builder Settings]] [[File:Eclipse ccpp build.png|500px|thumb|right|Figure 4. C/C++ Build ⇒ Builder Settings]] −[[File:Eclipse builders.png|500px|thumb|right|Figure 5. Project builders]]+[[File:Eclipse ccpp build behavior.png|500px|thumb|right|Figure 5. C/C++ Build ⇒ Behavior]] −[[File:Eclipse builders.png|500px|thumb|right|Figure 6. Project builders]]+[[File:Eclipse ccpp build refresh.png|500px|thumb|right|Figure 6. C/C++ Build ⇒ Refresh Policy]] −[[File:Eclipse builders.png|500px|thumb|right|Figure 6. Project builders]]+[[File:Eclipse source location.png|500px|thumb|right|Figure 7. C/C++ General ⇒ Paths and Symbols ⇒ Source Location]] Andrey.filippov

Oleg Dzhimiev committed changes to the Elphel git project :
gpio struct has changed in kernel 4.9