How Can I Add Git Version Info Into Cython-built .so File?

I want to add my git commit info (branch info and sha1 info) into my cython compiled out .so file. For example, I have a .pyx file, and I compile it to a .so module. but question

Solution 1:

The method by which you can embed a revision indicator into objects made from compiled source depends to great extent on the tools you use to compile the source, and of course on what source-code control system(s) you use to contain the source. The former determines what you can do and how you can run commands. The latter determines what information is important, which commands extract that information, and—to some extent—how to embed that information into a source file, if that can be done directly.

Since you asked specifically about Git, here are a few points:

• The branch name is essentially irrelevant. Git branch names mean almost nothing: a commit that is name-able via bleek today, might be named via pribble tomorrow.

• The output from git describe is more interesting, because it is unlikely to change, includes a nearby annotated tag—tags themselves are never supposed to change, though this requires that users making tags behave themselves—and gives you a convenient linear count of "distance from tag" plus a suffix that usually pinpoints a specific commit even without the rest of the information:

  v2.12.0-190-ge0688e9
  

  for instance is a mostly-readable name for a commit whose ID starts with e0688e9, and whose "nearby tag" is v2.12.0 (190 commits later, in some sense).

• The raw commit ID itself, in full, uniquely identifies the commit.

• It's possible to use the ident attribute as a filter—see the gitattributes documentation—to embed the raw commit ID in a source file. I would avoid this method myself unless the build system is too primitive to be able to run git describe.

[Edit: fixed this up a bit - you don't want to "import" the version file as a Python file, as that's done at runtime!]

You could, depending on your build system, have your builder do, e.g.:

echo'DESCRIBE = "' $(git describe --dirty) '"' > gitignored.pyx

and then build the file. The file itself should not be committed into the repository at all, but rather, built every time (it's always in need of re-generation).

This same method works for most compiled languages, though of course the form of the variable to define, and the syntax for strings and source files, varies from one to another.

Doing this directly in Cython, without an auxiliary file

For Cython in particular, if you use a setup.py, you can extract the Git version in Python code and pass it through as a -D argument during compilation. Here is an actual working example that modifies a basic setup.py to do this, formatted as a Git commit. The hello.pyx and setup.py files are nearly straight out of the initial Cython compilation example.

commit c1a008c1555be451047ff9869abe30c753cfc15d
Author: Chris Torek <chris.torek@gmail.com>
Date:   Wed Mar 15 02:03:41 2017 -0700

    build in Git version at compile time

diff --git a/hello.pyx b/hello.pyxindex da1b827..e3efef9 100644--- a/hello.pyx+++ b/hello.pyx@@ -1,2 +1,7 @@
 def say_hello_to(name):
     print("Hello %s!" % name)
++cdef extern from *:+    char *BUILD_VERSION++version = BUILD_VERSIONdiff --git a/setup.py b/setup.pyindex 6e6bc70..7c6d07c 100644--- a/setup.py+++ b/setup.py@@ -1,7 +1,23 @@
 from distutils.core import setup
 from Cython.Build import cythonize
+from Cython.Distutils.extension import Extension+from Cython.Distutils import build_ext++import subprocess+import sys++proc = subprocess.Popen(['git', 'describe', '--dirty'], stdout=subprocess.PIPE)+GIT_VERSION = proc.stdout.read().rstrip().encode('utf-8')+if proc.wait():+    sys.exit('git describe --dirty failed: exit code {}'.format(proc.wait()))++extensions = [+    Extension('hello', ['hello.pyx'],+        extra_compile_args=['-D', 'BUILD_VERSION="{}"'.format(GIT_VERSION)]),+]

 setup(
     name='Hello world app',
-    ext_modules=cythonize("hello.pyx"),+    cmdclass={'build_ext': build_ext},+    ext_modules=extensions
 )