You are currently viewing How to structure a project to be compilable by Arduino IDE and PlatformIO – Part 2

How to structure a project to be compilable by Arduino IDE and PlatformIO – Part 2

Arduino IDE and PlatformIO – Part 2 

How to structure a project to be compilable by both

Table of Contents

Introduction

Through the changes recommended by the first part of this article, we have reached to a point where our code is nicely separated and all compilers have access to all files. What could go wrong, right? Well, at the beginning of article we have made an assumption. Only header files were used for the sake of simplicity.

How CPP files could affect the folder structure

In a real world project, each header (.h) file would be accompanied by its corresponding source (.cpp) file. So our project source tree would be like this: 
senseAndUpload/
├── examples/
|   ├── exampleWifi/
|   │   └── exampleWifi.ino
|   └── exampleNBIoT/
|       └── exampleNBIoT.ino
└── src/
    ├── customLibs.h
    ├── device/
    |   ├── sensor.cpp
    |   └── sensor.h
    └── networking/
        ├── nbiot.cpp
        ├── nbiot.h
        ├── wifi.cpp
        └── wifi.h

Including a header file from a folder, all folder’s source files will be compiled

Issue number 5 which applies both for Arduino IDE and for PlatformIO. When including a header file from a folder, all folder’s .cpp files will be compiled and since the .cpp files include their corresponding .h headers, even the unused header files will be implicitly included. This happens because even though the compiler can determine which header files are included for a project, it is unable to determine automatically which source files to compile. This problem is solved in all typical software development tools that have project configuration with explicit declaration of the involved files, though Arduino IDE and PlatformIO do not have such support.

Let’s examine the effect of this behavior on the exampleWifi project. Imagine you have a device that is WiFi only, for example Arduino MKR1010. The sketch will be compiled having as target device the MKR1010. The compiler will successfully compile the “sensor.h“, “sensor.cpp“, “wifi.h“, “wifi.cpp” and will move on compiling “nbiot.cpp” which also includes the “nbiot.h“. This will throw a compilation error because the MKR1010 does not support NBIoT connectivity like MKR1500 so the compiler will not find all the needed libraries for NBIoT support. This leads us to the rule that a folder must contain code files that can all coexist when any of the folder’s header file is included.

Keeping this rule in mind we could make one folder for WiFi related code and one folder for NBIoT related code to isolate the functionality: 
senseAndUpload/
├── examples/
| ├── exampleWifi/
| │ └── exampleWifi.ino
| └── exampleNBIoT/
| └── exampleNBIoT.ino
└── src/
├── customLibs.h
├── device/
| ├── sensor.cpp
| └── sensor.h
└── networking/
├── nbiot/
| ├── nbiot.cpp
| └── nbiot.h
└── wifi/
├── wifi.cpp
└── wifi.h

Including 3rd level subfolder result to the compilation of all 3rd level local subfolders in PlatformIO

Keeping our focus on the exampleWifi project, which now includes the “wifi/wifi.h” file, the compilation in PlatformIO fails not being able to resolve dependencies of NBIoT. Wait, what? Even after our code separation and explicitly including a header file in a subfolder of networking subfolder, the code in “nbiot” is still being compiled. This reveals that PlatformIO selects the files to be compiled based on the direct subfolders of “src” folder regardless if there is further code separation within the subfolders.

To resolve this there are two approaches where only one of which works for both PlatformIO and Arduino IDE.

Put all code in “src” direct subfolders (PlatformIO only)

This solution would isolate the inclusion of all files, would work for PlatformIO though it could be quite limiting for large projects to have only one level of folder separation. Nevertheless, it would be acceptable in smaller projects: 
senseAndUpload/
├── examples/
|   ├── exampleWifi/
|   │   └── exampleWifi.ino
|   └── exampleNBIoT/
|       └── exampleNBIoT.ino
└── src/
    ├── customLibs.h
    ├── device/
    |   ├── sensor.cpp
    |   └── sensor.h
    ├── nbiot/
    |   ├── nbiot.cpp
    |   └── nbiot.h
    └── wifi/
        ├── wifi.cpp
        └── wifi.h

Use of target device macros

The above approach would not work for Arduino IDE because it is based on the include statements of “customLibs.h” file. The file contents would now be:
  • senseAndUpload/src/customLibs.h
#include "device/sensor.h"
#include "nbiot/nbiot.h"
#include "wifi/wifi.h"
While trying to compile Arduino IDE, the “exampleWifi.ino” would include “customLibs.h” thus it would include also the “nbiot.h“. This is an issue that we had from the beginning of our post though now it is time to solve it.

The cleanest approach would be to use the preprocessor macros defined when selecting a target device, to exclude code that is not compatible with the device capabilities. In our case lets assume we have the WiFi capable Arduino MRK1010 and the NBIoT capable Arduino MKR1500. When selecting each one for target device, a corresponding preprocessor macro gets enabled:
  • ARDUINO_SAMD_MKRWIFI1010
  • ARDUINO_SAMD_MKRNB1500
By wrapping around the contents of “nbiot.h” and “nbiot.cpp” with a directive ‘#ifdef ARDUINO_SAMD_MKRNB1500‘, the code is guarantied to be compiled only in case of MKR1500 device. Back in our example compilation of exampleWifi.ino, MKR1010 device will be selected, “nbiot.h” and “nbiot.cpp” will be included, though because of the macros, the preprocessor will exclude all the NBIoT specific code and the compilation will succeed.
Important step for PlatformIO
When following the approach of wrapping device-specific code around preprocessor macros, PlatformIO project configuration needs one additional configuration. PlatformIO has the LDF module (Library Dependency Finder) which runs before the compilation, looks up all the include statements from the project’s source files and decides which dependency libraries to use. The operation mode of LDF defaults to “chain”:

(from PlatformIO documentation)
chain: Parses ALL C/C++ source files of the project and follows only by nested includes (#include …, chain…) from the libraries. It also parses C, CC, CPP files from libraries which have the same name as included header file. Does not evaluate C/C++ Preprocessor conditional syntax.
Without the evaluation of the preprocessor macros, the LDF will detect both WiFi and NBIoT related code in the project, which is exactly what we were trying to avoid. To fix this, in the platformio.ino file add the statement: 
lib_ldf_mode = chain+
And finally everything works as expected.

Summary

As a project grows, so does the challenges that need to addressed. The addition of CPP files resulted to unpredicted changes that in cases required more advanced knowledge of the build mechanisms.

The final structure proposed by this article is: 
senseAndUpload/
├── examples/
| ├── exampleWifi/
| │ └── exampleWifi.ino
| └── exampleNBIoT/
| └── exampleNBIoT.ino
└── src/
├── customLibs.h
├── device/
| ├── sensor.cpp
| └── sensor.h
└── networking/
├── nbiot/
| ├── nbiot.cpp
| └── nbiot.h
└── wifi/
├── wifi.cpp
└── wifi.h
and here is the list of tool limitations that required special attention:
  • Generic
    • Including a header file in a folder, all folder’s source files will be compiled
    • use of preprocessor macros per device could be handy
  • PlatformIO only
    • including 3rd level subfolder result to the compilation of all 3rd level local subfolders
    • LDF mode must be set to “chain+” or “deep+” to evaluate preprocessor macros
The final version of this demo project can be downloaded here to be used as template for future projects.