Why I rewrote mkproj in shell script.

There are periods of time where I get like 5 different project ideas and really want to make them. So I sit down excited to start on a new project.

But then I immediately get stuck, trying to figure out how to configure my Makefile to run a simple "Hello, World!" program and track multiple source files, which quickly kills momentum.

To prevent this friction of boilerplate to start a project, there exist programs called project scaffolders which quickly help one to skip the boilerplate by generating all the boilerplate necessary. Many scaffolders already exist, but since I like making my own tools, I made mkproj.

Initial implementation of mkproj.

I initially envisioned mkproj to be configured using JSON. The idea was to have JSON files for each language, stored in ~/.config/mkproj. Along with that there would be separate JSON files for software licenses and documentation engines. Leading to the directory structure to look like:

% tree ~/.config/mkproj
~/.config/mkproj
├── c.json
├── documentation-engines.json
├── licenses.json
└── verilog.json

1 directory, 4 files

This should immediately raise some eyebrows. Using JSON files to define files is already bad design. Nonetheless, this version had parameter substitution for only ${PROJECT_NAME}, ${PROJECT_DESCRIPTION}, ${PROJECT_AUTHOR} and ${PROJECT_YEAR}.

Here is how a file was defined in mkproj:

{
    "filename": "src/main.c",
    "contents": ["int main()", "{", "    return 0;", "}", ""]
}

Now this looks normal, but I added some extra fields like:

insert/append: Insert and append some lines to filename AFTER all files have been made with their contents.
command: This is an interesting one, this basically runs a shell command.

Now there are options, which are composed of arrays of files. And templates which are composed of options. The full specification is given in the now archived repository for mkproj.

Since this project has many different components, including JSON parsing, running shell commands, editing/creating files; I decided to implement mkproj in Python.

Restructing the template formats.

Then I sat down to make templates, and I started feeling the friction of using JSON to describe file contents. However at the end I had a pretty nice set of 4 templates that I started to use.

Recently I got more into 'everything is a file' philosophy of ~~Unix~~ Plan 9. I realized two things: I wasn't really using the options (instead opting to use templates directly), and the only 3 core features necessary for a scaffolder would be:

Copying files to project directory.
Replacing special variables in the files (like ${PROJECT_NAME}).
Running shell commands.

I decided to write a more minimal version which does exactly this in C, but I pivoted to shell script because it is easier to handle files.

The final program.

The final program is about ~100 lines of shell script.

Here are some fun bits from the program...

UI.

The user interface is super simple. We get a list of possible things using ls -1, then store it in a variable. To display it, we pipe the variable through nl.

languages="$(ls -1 "$confpath")"
echo "$languages" | nl

Then we read a number from the user using read and get the correct line using sed.

echo -n 'language) '  # prompt for language
read lnr              # read into `lnr'
# sed -n suppresses the default printing of every line.
language="$(echo "$languages" | sed -n "${lnr}p")"

That is it for a fully functional UI! Of course the input isn't sanitised but that is because I did not want to overcomplicate a script only I use.

Running the `post` script.

Each template is just a directory that can be copied recursively to the desired destination (using cp -r). But to give more flexibility, I added this feature where if there is a post file present at the root of the template directory, then after copying it, it is executed then removed.

This is easily done using a subshell:

(
    cd "$project" || exit 1
    if [ -f post ]; then
        ./post "$project" "$description" "$author" || exit 1
    fi
    rm -f post
)

Licenses.

Licenses are super easy to manage. Licenses are stored in plain text, one license per file in a directory called license. The program first checks if the file COPYING exists already or not, then prompts the user to choose a license. This is useful when you have templates which already have a license.

Documentation.

Documentation is an important part of any project. There are many ways to write documentation, and these can be paired with the many templates. So it is better to have them separate.

mkproj stores documentation engine templates in the same way that it stores normal templates, complete with the post file support as well. To prevent mkproj from prompting to choose a documentation engine (because some templates can already have the documentation configured), a file named has_doc should be placed in the root of the template. This file is removed when mkproj successfully copies over the files. This is super simple to implement in shell script:

if [ ! -f "$project/has_doc" ]; then
    # ...
fi
rm -f "$project/has_doc"

To not choose any option for license or documentation, simply enter 0 when prompted.

This is simply implemented with a quick if:

if [ "$dnr" -ne "0" ]; then
    documentation="$(echo "$docs" | sed -n "${dnr}p")"
    cp -r "$docpath/$documentation/"* "$project/"
fi

Substitution of variables.

At the end of all this, the following command is run:

find . -type f -exec sed -i \
    -e "s|\${PROJECT}|$project|g" \
    -e "s|\${DESCRIPTION}|$description|g" \
    -e "s|\${AUTHOR}|$author|g" \
    -e "s|\${YEAR}|$year|g" {} +

This substitutes the special template variables in all the files (including binary files which may not be desirable), which was a requirement. This does not rename files containing the template string though, because that is a job for post.

Finally git init is run.

Templates.

Here are some cool tricks I used in post scripts for some templates. :)

Optional testing framework.

I wrote sht_test.h which is a simple single header testing framework for C. Since I don't want it for every project, I just used post to ask me if I want it or not then directly fetch it using wget, and also make the main testing entry point file using cat <<EOF >tests/test.c.

Optionally add targets to Makefile if present.

I like using Makefile as a way to build everything, including documentations. Since a Makefile may or may not be present for every project, it is wiser to check if said Makefile exits and then just append to it.

This keeps it clean.

README.txt heading formatting.

I have started using README.txt more than markdown and typically write the project heading as:

sample project
==============
sample text sample text...

The heading has an equal amount of = signs underneath it. How would I make this happen (because the project name, and hence the number of characters in it are not known until mkproj is run) for any project name?

The answer is this clever line of sed:

sed -i -e '1p' -e '1s/./=/g' README.txt

What this does is print the 1st line 1p and then prints the 1st line but this time any character is replaced with = (1s/./=/g).

Finally...

I ended up archiving the Python version of mkproj since this new version is much better, it also makes me look smart :P.

Now I just can quickly make new templates without writing JSON and speed up my ideas to projects workflow... hopefully.