This document describes the modular structure of the cuPoisson library.
It will be useful for people developing the cuPoisson library, but not for library users.
Firstly, the module concept is explained and then, the organization of the source files is described.

Modules
------- 

The library is composed by a kernel ---or core--- and several optional modules.
This way, the user can decide which functionality will be built and added to its final library version.

Currently there are 2 modules ---Serial and MPI.
At least one module must be included in the build, since the library core does not provide any useful functionality by itself.
The Serial module is considered the default module, but it can be disabled if at least another one is enabled.

The user will select the modules he wants to build when configuring the project with CMake.
The selection of a module has the effect of defining a C Macro for the library programmer ---MOD_SERIAL and MOD_MPI--- and another one for the library user ---CUP_MOD_SERIAL and CUP_MOD_MPI.

The library core will comprise three kind of symbols:

· Symbols exported for the final user.
They will all be prefixed with 'cup_' and will be included in the main "cuPoisson.h" header file.
These symbols will be general symbols that cannot be related to a particular module or Poisson solver.
Some examples are functions such as cup_get_valid_devices(), cup_get_error_string() or structures such as cup_grid_t.

· Public general symbols to be used by the rest of the modules.
This definition comprises basically utility functions such as basic linear algebra functions,  basic data communication functions and so on.

· Private symbols used to implement the previously described ones.

The modules will comprise two kind of symbols:

· Symbols exported for the final user.
They will all be prefixed with 'cup_' and the module prefix, i.e. 'cup_mpi_'.
For simplicity, the default Serial module's exported symbols will lack the module prefix. 
These symbols will be declared in the module header files, such as "cuPoisson_ser.h" and "cuPoisson_mpi.h".
The main header file "cuPoisson.h" will include the module header files of the modules built.
These symbols will be closely related to a particular implementation of a Poisson solver, e.g. cup_mpi_poisson_cg3d.

· Private symbols that will not be exported and are used to implement the previously described ones.
In this case, even the Serial module symbols will be prefixed by the module prefix, i.e. ser_, but not by the library prefix (cup_) which is reserved for exported symbols.
This incoherence looks for regularity in the internal code, and simplicity for the novice user. 

File structure
--------------

The compilation of each source file is controlled by C preprocessor macros and conditional directives.
The code run from a single module must be enclosed by a #ifdef directive, such as "#ifdef MOD_MPI [...] #endif".
The code that corresponds to several, but not all, modules must be enclosed by a #if directive, such as, "#if defined MOD_SER || defined MOD_MPI [...] #endif".
The code used from every module will be part of the library core and must be coded directly, with no module related conditional directive.
These directives allow reducing the library size, since just the code corresponding to the selected modules is included in it.

If the code is organized in files according to its module membership the conditional directives can be avoided and the inclusion/exclusion in the library controled by CMake.
However, while the amount of files in the project is low, I prefer organizing the code in files according to its functionality rather than its module membership.

Sometimes, two or more modules will have different, but very similar, functions, e.g. transfer_ghost_points() for the SERIAL and MPI modules.
In this case, CMake will build several objects with all the implementations of the functions.
I think the best way to explain how this should work is with a few examples.

Let's assume that our library has 3 modules ---Serial, MPI and Other--- controlled by the MOD_SER, MOD_MPI and MOD_OTHER macros.
Let's assume that there is a function called 'some_function()' that works for MOD_SER and MOD_OTHER, but we must write it for MOD_MPI.
The function corresponding to MOD_MPI is very similar to the original one and therefore, we use the same code.
We will leave a 'generic' version of the function called 'some_function()' in the library core, and create a new one called 'mpi_some_function()' that will be used from the MPI module.
We can do that declaring the function as:

#ifdef MOD_EXCL_MPI
void mpi_some_function()
#else
void some_function()
#endif

And then, adding some "#ifdef MOD_EXCL_MPI [...] #endif" declarations in the function body where code corresponding to just the MPI module is found.

CMake will have to be configured to build two versions of this function, called 'some_function()' and 'mpi_some_function()'.
This will be done adding the file name to the SRC and MULTI_SRC_MPI variables of the CMakeLists.txt file.
For simplicity we will keep this "mixed" files separate from code that belongs to a single module.

We can modify the previous example if the same function has different variations for each module and therefore, there is not a generic module that belongs to the library core.
In this case CMake should be configured accordingly and the function could be declared as:

#if defined MOD_EXCL_MPI
void mpi_some_function()
#elif defined MOD_EXCL_SER
void ser_some_function()
#elif defined MOD_EXCL_OTHER
void other_some_function()
#endif

or, since all MOD_EXCL_* macros will be exclusive,

#ifdef MOD_EXCL_MPI
void mpi_some_function()
#endif
#ifdef defined MOD_EXCL_SER
void ser_some_function()
#endif
#ifdef defined MOD_EXCL_OTHER
void other_some_function()
#endif

In this case, the file name cannot be added to the SRC variable in CMake, but to the MULTI_SRC_SERIAL, MULTI_SRC_MPI and MULTI_SRC_OTHER variables.

The source files will be structured in folders regarding to its functionality and not its corresponding module.
Typical folders are 'utils' for utility functions, 'core' for functions of the library core or 'solvers' for Poisson solvers.
Files that must generate several object files will be identified by the '_mods' suffix.
If a big source file must be split in smaller files and module membership is a reasonable criteron, a suffix should be used to identify the module.
