閱讀679 返回首頁    go 阿裏雲 go 技術社區[雲棲]


History of UNIX Project Build Tools

版權聲明:本文為博主原創文章,未經博主允許不得轉載。

History of UNIX Project Build Tools  

(The following is derived from the HACKING.txt file of the old open source project which I stopped supporting many years ago. R.I.P.)

You might have noticed above that there are SIX STEPS required to do a rebuild after editing configure.in. Why is it so complicated?

最初最初的時候,隻有Makefile,並沒有configure等工具出現。

You might remember the days when all the dependencies and rules were encapsulated in one file (called Makefile) and no matter what you changed (including the Makefile itself) the make command would figure out how to rebuild everything. That's not true anymore.

Unix版本越來越多,開始要求Makefile具備跨平台的功能:不同平台,對Makefile有不同的要求。這時工具鏈就必須延伸。

The original Makefile model worked well before the proliferation of many different types of UNIX and the advent of cross-platform compatibility.

To handle all the different types of Unix, the Makefile has to be complex — so complex that it is no longer practical to edit by hand. Furthermore, many things have to be figured out by the machine where the program is going to be built, rather than the machine where the programmer developed it. So, the "meta-rules" for creating Makefiles got too complicated to edit by hand. Eventually there got to be a lot of different types of source files, and a lot of different rules for what to do if you want to change something. The best way to explain this is by going through the history in chronological order — starting prior to the origin of Makefile and make.

為了理解工具鏈是如何一步步延伸的,就從曆史先後順序看起。

人肉CC搞定

Originally, all changes were made just by changing C source code (in the foo.c and foo.h files) and typing cc to compile it into a binary:

sources -> cc -> binary

用Makefile推導文件依賴,自動編譯

Then came compiling and linking as a separate step. This created the situation that if you change just one foo.c file, you only have to re-compile that one file and then link, but if you change a foo.h file you probably had to compile everything. To save time, people figured out how to make a list of rules telling which ".c" files depend on which ".h" files. The make tool was developed, a program that would automatically figure out what needed to be recompiled. make takes a new source file, called Makefile, and also uses all the program source files. Rebuilding still consisted of just one step:

Makefile and sources -> make -> binary

make wasn't quite as smart as it should have been. For example, there's no way to get it to check if Makefile itself was changed. To work around this, programmers started adding a "target" called "clean" (or something similar) that removes all the object files. Then, if you change Makefile (for example, after realizing you left out an "#include" dependency) you can type make clean to force it to remove the objects, and make to recompile everything. Commands like make clean are still used today. (花絮,這裏解釋了為什麼改變了Makefile後應該執行make clean)

多種Unix變種出現,如BSD,AT&T System III,源碼和Makefile中需要條件編譯/配置(configuration),以適應不同係統。早起做法是把配置集中到一個頭文件裏麵,例如config.h中,或放到Makefile裏。

After a few years, different versions of Unix started to exist (like BSD vs. AT&T System III), and people noticed that you had to change the foo.c and foo.h files and Makefile in different ways depending on what type of Unix you were compiling for. Those changes were called "configuration". Manual configuration is tedious — it takes a lot of knowledge and diligence to make all the correct changes for your own particular type of Unix. Eventually it was decided all such changes should be controlled by #ifdef tests (like "|#ifdef BSD_4_1"), and the #defines could be specified in the Makefile| or a header file called config.h (or something similar). Building then required two steps. In the first step, you edit the Makefile to #define each of the defines (like BSD_4_1) that you need on your system:

[plain] view plaincopy
  1.           generic  
  2.         Makefile and  -.  
  3.           config.h      \  
  4. STEP 1.             manually-edit  
  5.                           \  
  6.                            `->  custom Makefile  
  7.                                  and config.h  
  8.   
  9.             Makefile  
  10.                  and   -.  
  11.              sources     \  
  12. STEP 2.                 make  
  13.                           \  
  14.                            `->  binary  


然後大家受不了這麼山寨的搞法,發明了專門的配置係統(1992年),它首先探測係統類型,然後生成針對這個係統的一坨宏。這個係統的名字叫configure (Makefile的上一步出現!),它是個shell腳本。


Next, standard "configuration systems" were created. Usually a configuration system was a set of shell scripts that made all the tests and modifications automatically. For example, it is easy to test for BSD version 4.1, and everyone agreed to use BSD_4_1 to indicate you're on that system. So, (in theory) all it takes is one big set of instructions, (typically, a shell script called configure), to test for all the different types of hardware, oeprating systems, libraries, etc. and generate the #defines for that system. The result, for most programmers, was to replace the first manual step with something more automatic.

Because Makefile was now auto-generated by configure, the configure file was what you edited when you wanted to change the Makefile, and the Makefile became an uneditable, automatically-generated file just like the program binary. The two build steps became:


[plain] view plaincopy
  1.  configuration-files  -.  
  2.                         \  
  3. STEP 1.              configure  
  4.                           \  
  5.                            `->  Makefile  
  6.   
  7.             Makefile  
  8.                  and   -.  
  9.              sources     \  
  10. STEP 2.                 make  
  11.                           \  
  12.                            `->  binary  



起初configure要負責兩件事:探測操作係統類型;生成Makefile。後來(1994年),這兩件事分化為由兩個工具來完成:autoconf負責探測操作係統類型;configure負責生成Makefile。autoconf的參數文件叫configure.in;configure的參數文件叫Makefile.in。


Several different types of configuration systems were in place by 1992. Some consisted of a script called configure that did all the tests to see what type of Unix you're running on, then generated the Makefiles. The configure script had to know a lot about the syntax of makefiles, as well as knowing a lot about how to test for different features of operating systems.

Eventually, the job of doing the operating-system tests and the job of creating the Makefiles from "Makefile templates" was split up into two different tools.

By 1994 it was generally agreed that the best tool for the operating-system tests was autoconf. It took one new source file: configure.in and generated a script called configure as output. This configure script, in turn, took one new source file called Makefile.in, and generated Makefile as an output file. At this point the build had three steps that worked like this:


[plain] view plaincopy
  1.           configure.in -.  
  2.                          \  
  3. STEP 0.               autoconf  
  4.                           \  
  5.                            `->  configure  
  6.   
  7. - - - - tarfile is distributed in this form - - - -  
  8.   
  9.           Makefile.in -.  
  10.                         \  
  11. STEP 1.              configure  
  12.                           \  
  13.                            `->  Makefile  
  14.   
  15.             Makefile  
  16.                  and   -.  
  17.              sources     \  
  18. STEP 2.                 make  
  19.                           \  
  20.                            `->  binary  


第一步用autoconf生成configure文件這件事一般隻需要做一次,因為係統不會總是變,也不會經常增加對操作係統有新依賴的代碼。回想一下,我們下載的很多開源包裏麵,隻需要執行./configure; make; make install就行了,並沒有autoconf什麼事,就是這個原因。

Note that Step 0 only had to be done if you changed the configuration requirements, like if you added a major new feature that depended on something that is different on different systems (an example would be adding a graphical user interface to a program that was previously text-only). Therefore, the build process was now split into the "user installation" steps (steps 1 and 2) and the "complete rebuild from scratch" (steps 0 1 and 2). Typically, the programmer would perform step 0 and distribute the result to the users, who perform steps 1 and 2. This is indicated above where it says "tarfile is distributed in this form".

到autoconf這一溫飽時代後,人們開始奔小康,對方便性提出了更高的要求:configure的輸入文件Makefile.in太他媽大了啊!工程下麵的所有文件都要寫到這裏麵去,維護起來很費勁(不信你去找個Makefile.in來看看)。於是,大約在1996年,又發明了一個新工具:automake,專門用於生成Makefile.in。automake的輸入文件叫Makefile.am。

The weak point in this system was Makefile.in. This had to be a very large and complex file, because it contained all the rules for how to generate a Makefile, and Makefiles were by this point very complex (about as complex as a programming language) and vary a lot from one OS to another. Since Makefile.in was a source file it had to be edited manually. Most of Makefile.in was the same regardless of what program you were building, and programmers found it cumbersome.

The solution to that was automake. It automatically creates Makefile.in from another new source file, called Makefile.am. By 1996, the standard build process had four steps (two for users doing an install and two more for people adding new features) and the steps were:


對於代碼維護者,需要涉及到下圖中4步;對於開源代碼使用者,隻需要涉及到後麵兩步。


[plain] view plaincopy
  1.           configure.in -.  
  2.                          \  
  3. STEP 0-A.             autoconf  
  4.                           \  
  5.                            `->  configure  
  6.   
  7.            Makefile.am -.  
  8.                          \  
  9. STEP 0-B.             automake  
  10.                           \  
  11.                            `->  Makefile.in  
  12.   
  13. - - - - tarfile is distributed in this form - - - -  
  14.   
  15.           Makefile.in -.  
  16.                         \  
  17. STEP 1.              configure  
  18.                           \  
  19.                            `->  Makefile  
  20.   
  21.             Makefile  
  22.                  and   -.  
  23.              sources     \  
  24. STEP 2.                 make  
  25.                           \  
  26.                            `->  binary  


插播一個問題:configure.ac在哪裏?答:還沒出生呢!

還記得configure.in是怎麼來的嗎?當年為了給configure減負,把操作係統探測的邏輯獨立出來,搞了個autoconf工具。autoconf工具的輸入文件叫configure.in。又過了幾年,configure.in成了瓶頸,太他媽難寫了,不光操作係統,什麼庫啊,驅動啊,都得管。為了解決這個問題,autoconf擴展了自己的功能,引入了一個叫做aclocal.m4的”宏文件“,這個文件是用一種叫”m4”的語言寫的,擅長探測操作係統的方方麵麵。擴展後的autoconf,組合configure.in和aclocal.m4為輸入,生成configure文件。

Over the next couple years, configure.in got bigger and included lots of code to test for lots of different types of libraries, drivers, operating systems, etc. Eventually configure.in became the biggest and hardest-to-maintain file, just like Makefile.in had been. More recent versions of autoconf have solved this by allowing for the use of a "macros" file called aclocal.m4. The "macros" are written in a language called m4, and they contain the rules for performing all sorts of different operating-system tests. As far as the build process is concerned, these can be treated as part of step 0-A, except that you don't ever have to worry about changing the contents of aclocal.m4:


[plain] view plaincopy
  1.           configure.in   
  2.             aclocal.m4   -.  
  3.                            \  
  4. STEP 0-A.               autoconf  
  5.                             \  
  6.                              `->  configure  
  7.   
  8. STEP 0-B.  (automake step, same as above)  
  9.   
  10. - - - - tarfile is distributed in this form - - - -  
  11.   
  12. STEP 1.    (configure step, same as above)  
  13.   
  14. STEP 2.    (make step, same as above)  


aclocal.m4可用aclocal工具直接生成,也可以手寫。如果用工具生成的話,就需要引入macros文件作為aclocal工具的輸入。

Around the same time it also became common to use a tool called aclocal to generate aclocal.m4, from a directory of macros files called "macros". This added a fifth step to the full build process:

[plain] view plaincopy
  1.           configure.in   
  2.            macros/*.m4   -.  
  3.                            \  
  4. STEP 0-A.            aclocal -I macros  
  5.                              \  
  6.                               `->  aclocal.m4  
  7.   
  8.           configure.in   
  9.             aclocal.m4   -.  
  10.                            \  
  11. STEP 0-B.               autoconf  
  12.                             \  
  13.                              `->  configure  
  14.   
  15. STEP 0-C.  (automake step, same as above)  
  16.   
  17. - - - - tarfile is distributed in this form - - - -  
  18.   
  19. STEP 1.    (configure step, same as above)  
  20.   
  21. STEP 2.    (make step, same as above)  


當新千年的鍾聲想起,我們終於告一段落,大廈落成。

This was the way things were done by around the year 2000.

Complete list of files and the order in which they are built:


[plain] view plaincopy
  1. ORIGINAL FILES  
  2.    
  3.         the file: configure.in  
  4.  is created from: typed in by hand  
  5.    
  6.         the file: Makefile.am  
  7.  is created from: typed in by hand  
  8.    
  9.         the file: src/adam.c  
  10.  is created from: typed in by hand  
  11.    
  12.         the file: src/adam.h  
  13.  is created from: typed in by hand  
  14.    
  15.         the file: src/anything.c      (any ".c" not listed below)  
  16.  is created from: typed in by hand  
  17.    
  18.         the file: src/anything.h      (any ".h" not listed below)  
  19.  is created from: typed in by hand  
  20.    
  21.    
  22.    
  23.  AUTO_GENERATED FILES  
  24.    
  25.         the file: config.h.in   
  26.  is created from: acconfig.h configure.in acconfig.h  
  27.               by: autoheader  
  28.    
  29.         the file: config.h  
  30.  is created from: config.h.in  
  31.               by: ./configure  
  32.    
  33.         the file: Makefile  
  34.  is created from: Makefile.in  
  35.               by: ./configure  
  36.    
  37.         the file: configure  
  38.  is created from: configure.in aclocal.m4  
  39.               by: autoconf  
  40.    
  41.         the file: aclocal.m4  
  42.  is created from: configure.in macros/*.m4  
  43.               by: aclocal -I macros  
  44.    
  45.         the file: Makefile.in  
  46.  is created from: Makefile.am  
  47.               by: automake  
  48.    

不過,還是那個問題:configure.ac在哪呢?configure.in還得手寫?我嘞個去!

表著急,configure.ac隻不過是configure.in的新名字而已啦:

At this time aclocal and AM_INIT_AUTOMAKE did not exist, so many things had to be done by hand. For instance, here is what a configure.in (this is the former name of the configure.ac we use today) must contain in order to use Automake 0.20:


原文地址:https://mrob.com/pub/comp/unix-building-history.html


補充閱讀:

autoconf的曆史:https://www.gnu.org/software/autoconf/manual/autoconf-2.65/html_node/History.html

automake的曆史:ttps://www.gnu.org/software/automake/history/automake-history.html#Timeline

整個流程可視化:https://en.wikipedia.org/wiki/Configure_script


特別推薦看看automake的曆史,裏麵有數位作者的回憶,從中可以了解到很多原委。

最後更新:2017-04-01 13:37:10

  上一篇:go lucene如何通過docId快速查找field字段以及最近距離等信息?
  下一篇:go 平台化三部曲之一微核心可擴展架構 - 從Eclipse平台看交易平台化