内存管理pbuf.c源码解析——LwIP学习
声明：个人所写所有博客均为自己在学习中的记录与感想，或为在学习中总结他人学习成果，但因本人才疏学浅，如果大家在阅读过程中发现错误，欢迎大家指正。
本文自己尚有认为写的不完整的地方，源代码没有完全理清，以后会不定期更新
上一篇分析了pbuf.h头文件，这次来分析LwIP的内核(core文件夹)中的pbuf.c源代码。本人使用的LwIP源代码为Lwip-1.4.1版本。
pbuf.h文件分析地址：http://blog.csdn.net/angel_94/article/details/50111163
一、关于pbuf.c
LwIP的内核(core文件夹)文件中pbuf.c是包含协议栈内核使用的数据包管理函数，用于协议栈层次间的数据传递，避免数据拷贝。我们来分析pbuf.c源代码。
其中主要包含的函数：
pbuf_alloc()：内存申请函数
pbuf_realloc()：调整收缩pbuf的大小，在相应pbuf(链表)尾部释放一定的空间，将数据包pbuf中的数据长度减少为某个长度值
pbuf_header()：调整payload指针和长度字段以便为pbuf中的数据预置包头，常用于实现对pbuf预留孔间的操作
pbuf_free()：数据包释放函数
pbuf_ref()：用于将pbuf中的ref加1
pbuf_chain()：用于连接pbufs，连接两个pbuf(链表)为一个pbuf链表
pbuf_dechain()：用于连接pbufs
pbuf_copy()：用于将一个任何类型的pbuf中的数据拷贝到一个PBUf_RAM类型的pbuf中
pbuf_take()：用于向pbuf的数据区域拷贝数据
二、内存申请函数 pbuf_alloc()
1、关于pbuf
内存申请函数是这个文件中最重要的函数
pbuf是LwIP信息包的内部表示，为最小限度协议栈的特殊需求而设计。pbufs与BSD实现中使用的mbufs相似。pbuf结构即支持动态内存分配保存信息包内容，也支持让信息包数据驻留在静态存储区。pbufs可以在一个链表中链接在一起，被称为一个pbuf链，这样一个信息包可以穿越几个pbufs。
pbufs有四种类型：PBUF_RAM、PBUF_ROM、PBUF_REF、PBUF_POOL。
程序经常使用LWIP_DEBUGF()函数：LWIP_DEBUGF()是LwIP协议栈的调试信息输出函数
1.可以查看函数的调用关系，跟踪程序流程
2.查看各种协议的调试信息，关键变量的值
2、pbuf_alloc()函数
/**
* Allocates a pbuf of the given type (possibly a chain for PBUF_POOL type).
*分配一个给定类型(可能是一个PBUF_POOL型的链)的pbuf
*
* The actual memory allocated for the pbuf is determined by the
* layer at which the pbuf is allocated and the requested size
* (from the size parameter).
*实际内存分配的pbuf是由分配的pbuf的层次和请求的大小决定的
*
* @param layer flag to define header size
* @param length size of the pbuf's payload
* @param type this parameter decides how and where the pbuf
* should be allocated as follows:
*
* - PBUF_RAM: buffer memory for pbuf is allocated as one large
* chunk. This includes protocol headers as well.
* PBUF_RAM：为pbuf缓冲存储器分配一大块。这也包括协议头
*
* - PBUF_ROM: no buffer memory is allocated for the pbuf, even for
* protocol headers. Additional headers must be prepended
* by allocating another pbuf and chain in to the front of
* the ROM pbuf. It is assumed that the memory used is really
* similar to ROM in that it is immutable and will not be
* changed. Memory which is dynamic should generally not
*没有缓冲内存分配给PBUF，即使是协议头。附加协议头必须预先在到ROM中的pbuf之前分配另一个pbuf和链表考虑
*假定内存使用的内存和ROM是非常相似的，它是不可改变的而且不会改变。
*内存是动态的，一般不应附着到PBUF_ROM中的pbuf。使用PBUF_REF相反。
*
* - PBUF_REF: no buffer memory is allocated for the pbuf, even for
* protocol headers. It is assumed that the pbuf is only
* being used in a single thread. If the pbuf gets queued,
* then pbuf_take should be called to copy the buffer.
*没有缓冲内存分配给PBUF，即使是协议头。假定这个pbuf只被使用在单个线程中。
*如果pbuf被排队，然后pbuf_take应该叫做复制缓冲区
*
* - PBUF_POOL: the pbuf is allocated as a pbuf chain, with pbufs from
* the pbuf pool that is allocated during pbuf_init().
*PBUF_POOL：所述的pbuf被分配为pbuf链，与来自pbuf池的pbufs在pbuf_init()即内存初始化函数中被分配
*
*
* @return the allocated pbuf. If multiple pbufs where allocated, this
* is the first pbuf of a pbuf chain.
* 返回被分配的pbuf。如果多个pbuf被分配，这是pbuf链中的第一个pbuf
*/
struct pbuf *
pbuf_alloc(pbuf_layer layer, u16_t length, pbuf_type type)
{
struct pbuf *p,*q, *r;
u16_t offset; /*有效数据起始偏移位置*/
s32_t rem_len; /* remaining length 剩余长度*/
LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_TRACE,("pbuf_alloc(length = %"U16_F")\n", length);
/* determine header offset */
seitch(layer) { /* pbuf的层次 */
case PBUF_TRANSPORT:
/* add room for transport (often TCP) layer header */
/* 如果为传输层申请pbuf，那么有效数据的偏移位置为PBUF_LINK_HLEN +PBUF_IP_HLEN +PBUF_TRANSPPORT_HLEN */
offset = PBUF_LINK_HLEN + PBUF_IP_HLEN + PBUF_TRANSPPORT_HLEN;
break；
case PBUF_IP:
/* add room for IP layer header */
/* 如果为ip层申请pbuf，那么有效数据的偏移位置为PBUF_ip_HLEN + PBUF_LINK_HLEN */
offset= PBUF_LINK_HLEN + PBUF_IP_HLEN;
case PBUF_LINK:
/* add room for link layer header */
/* 如果是链路层申请pbuf内存，那么有效数据的偏移位置为PBUF_LINK_HLEN */
offset = PBUF_LINK_HLEN;
break;
case PBUF_RAM:
/*如果为原始层申请pbuf内存，那么数据偏移位置就是0，不预留任何空间*/
offset= 0；
break;
default：
LWIP_ASSERT("pbuf_alloc:bad pbuf layer", 0);
return NULL;
}
/********************************************************/
switch (type){ /* pbuf的类型 */
case PBUF_POOL: /*PBUF_POOL分配一个链表，链表上每个元素所管理的内存最大不超过PBUF_POOL_BUFSIZE*/
/* allocate head of pbuf chain into p */
p = (struct pbuf *)memep_malloc(MEMP_PBUF_POOL);/* 为PBUF_POOL类型时 */
/*PBUF_POOL类型和PBUF_RAM类型的都由内存池分配得到，此处调用了memp_malloc，在memp.c文件中 */
/* 分配的内存池类型为MEMP_PBUF_POOL*/
LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_alloc: allocated pbuf %p\n"), (void *)p)); /* p是pbuf类型的指针*/
/*LWIP_DEBUGF是LwIP协议栈的调试信息输出函数 */
/*1.可以查看函数的调用关系，跟踪程序流程。*/
/*2.查看各种协议的调试信息，关键变量的值*/
if(p == NULL){
PBUF_POOL_IS_EMPTY();
return NULL;
}
p->type = type;
p->next = NULL;
/* make the payload pointer point 'offset' bytes into pbuf data memory */
p->payload = LWIP_MEM_ALIGN((void *)((u8_t *)p + (SIZEOF_STRUCT_PBUF + offset)));
/* payload指向pbuf管理的数据的起始地址，payload指向的起始地址即为offset。*/
/*以offset为基准，登记有效数据存储的起始偏移位置到p->payload[luther.gliethttp] */
LWIP_ASSERT("pbuf_alloc: pbuf p->payload properly aligned",
((mem_ptr_t)p->payload % MEM_ALIGNMENT) == 0);
/* the total length of the pbuf chain is the requested size */
p->tot_len = length; /* pbuf链表上有效数据总大小 */
/* set the length of the first pbuf in the chain */
p->len = LWIP_MIN(length, PBUF_POOL_BUFSIZE_ALIGNED - LWIP_MEM_ALIGN_SIZE(offset));
/*pbuf链表上的每个元素所能存储的最大数据为PBUF_POOL_BUFSIZE。如果超过，如果超过该值，那么就会使用链表方式。*/
/*链接其他很多个pbuf，直到申请的size数据全部能够正常存储为止[luther.gliethttp]*/
LWIP_ASSERT("check p->payload + p->len does not overflow pbuf",
((u8_t*)p->payload + p->len <=
(u8_t*)p + SIZEOF_STRUCT_PBUF + PBUF_POOL_BUFSIZE_ALIGNED));
LWIP_ASSERT("PBUF_POOL_BUFSIZE must be bigger than MEM_ALIGNMENT",
(PBUF_POOL_BUFSIZE_ALIGNED - LWIP_MEM_ALIGN_SIZE(offset)) > 0 );
/* set reference count (needed here in case we fail) */
p->ref = 1; /* ref表示当前pbuf被引用的次数 */
/* now allocate the tail of the pbuf chain */
/* remember first pbuf for linkage in next iteration */
r = p;
/* remaining length to be allocated */
rem_len = length - p->len;
/* any remaining pbufs to be allocated? */
while (rem_len > 0) {
q = (struct pbuf *)memp_malloc(MEMP_PBUF_POOL);
if (q == NULL) {
PBUF_POOL_IS_EMPTY();
/* free chain so far allocated */
pbuf_free(p);
/* bail out unsuccesfully */
return NULL;
}
q->type = type;
q->flags = 0;
q->next = NULL;
/* make previous pbuf point to this pbuf */
r->next = q;
/* set total length of this pbuf and next in chain */
LWIP_ASSERT("rem_len < max_u16_t", rem_len < 0xffff);
q->tot_len = (u16_t)rem_len;
/* this pbuf length is pool size, unless smaller sized tail */
q->len = LWIP_MIN((u16_t)rem_len, PBUF_POOL_BUFSIZE_ALIGNED);
q->payload = (void *)((u8_t *)q + SIZEOF_STRUCT_PBUF);
LWIP_ASSERT("pbuf_alloc: pbuf q->payload properly aligned",
((mem_ptr_t)q->payload % MEM_ALIGNMENT) == 0);
LWIP_ASSERT("check p->payload + p->len does not overflow pbuf",
((u8_t*)p->payload + p->len <=
(u8_t*)p + SIZEOF_STRUCT_PBUF + PBUF_POOL_BUFSIZE_ALIGNED));
q->ref = 1;
/* calculate remaining length to be allocated */
rem_len -= q->len;
/* remember this pbuf for linkage in next iteration */
r = q;
}
/* end of chain */
/*r->next = NULL;*/
break;
/********************************************************/
case PBUF_RAM: /*PBUF类型内存，一次性分配size大小的连续内存*/
/* If pbuf is to be allocated in RAM, allocate memory for it. */
p = (struct pbuf*)mem_malloc(LWIP_MEM_ALIGN_SIZE(SIZEOF_STRUCT_PBUF + offset) + LWIP_MEM_ALIGN_SIZE(length));
/*PBUF_RAM由内存堆分配内存，此处调用了mem_malloc()，为内存堆分配函数，在mem.c文件中。*/
/*分配的空间大小包括：pbuf结构头大小size_struct_pbuf，需要的数据存储空间大小length。*/
/*还有一个offset，即有效数据起始偏移位置 */
if (p == NULL) {
return NULL;
}
/* Set up internal structure of the pbuf. */
p->payload = LWIP_MEM_ALIGN((void *)((u8_t *)p + SIZEOF_STRUCT_PBUF + offset));
p->len = p->tot_len = length;
p->next = NULL;
p->type = type;
LWIP_ASSERT("pbuf_alloc: pbuf->payload properly aligned",
((mem_ptr_t)p->payload % MEM_ALIGNMENT) == 0);
break;
/********************************************************/
/* pbuf references existing (non-volatile static constant) ROM payload? */
case PBUF_ROM: /*ROM只需要分配小小的管理pbuf的控制管理内存*/
/* pbuf references existing (externally allocated) RAM payload? */
case PBUF_REF:
/* only allocate memory for the pbuf structure */
/*只需申请pbufs头控制结构体所需内存即可[luther.gliethttp]*/
p = (struct pbuf *)memp_malloc(MEMP_PBUF);
/*分配的内存池类型为MEMP_PBUF，大小恰为一个pbuf头的大小*/
if (p == NULL) {
LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_LEVEL_SERIOUS,
("pbuf_alloc: Could not allocate MEMP_PBUF for PBUF_%s.\n",
(type == PBUF_ROM) ? "ROM" : "REF"));
return NULL;
}
/* caller must set this field properly, afterwards */
p->payload = NULL;
p->len = p->tot_len = length;
p->next = NULL;
p->type = type;
break;
/********************************************************/
default:
LWIP_ASSERT("pbuf_alloc: erroneous type", 0);
return NULL;
}
/* set reference count */
p->ref = 1;
/* set flags */
p->flags = 0;
LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_alloc(length=%"U16_F") == %p\n", length, (void *)p));
return p;
}三、连接pbufs
连接pbufs主要使用了三个函数：pbuf_cat()、pbuf_ref()、pbuf_chain()。
1、pbuf_ref()：将 pbuf中的 ref 加1
/**
* Increment the reference count of the pbuf
* 增量的pbuf的引用次数
* @param p pbuf to increase reference counter of
* 参数p的pbuf引用计数增加
* /
void
pbuf_ref(struct pbuf *p)
{
SYS_ARCH_DECL_PROTECT(old_level);
/* pbuf given? */
if(p != NULL) {
SYS_ARCH_PROTECT(old_level);
++(p->ref); /*当pbuf不为空时将当前pbuf的ref加1*/
SYS_ARCH_UNPROTECT(old_level);
}
}ref是pbuf结构中定义的一个16位无符号整数，包含一个引用计数，表示该pbuf被引用的次数，初始化一个pbuf的时候，ref字段值被设置为1，当有其他pbuf的next值针指向该pbuf时，该pbuf的字段值加1，所以要删除一个pbuf时，ref的值必须为1才能删除成功，否则删除失败。 统计有多少个指针指向这个pbuf。这些指针可能是应用程序的指针，协议栈自己的指针或者数据链中的pbuf->next指针，ref为0时，才可以释放pbuf
2、pbuf_cat()
/**
* Concatenate two pbufs (each may be a pbuf chain) and take over
* the caller's reference of the tail pbuf.
* 连接两个pbufs(每个可能都是链表)并替代引用的呼叫方的尾部pbuf
* @note The caller MAY NOT reference the tail pbuf afterwards.
* 注意对方可能之后不提及尾部的pbuf
* Use pbuf_chain() for that purpose.
* 目的是使用pbuf_chain()，在pbuf_chain()之中使用这个函数来连接两个pbufs
* @see pbuf_chain()
*/
void
pbuf_cat(struct pbuf *h, struct pbuf *t)
{
struct pbuf *p; /* p最为一个中间变量 */
LWIP_ERROR("(h != NULL) && (t != NULL) (programmer violates API)",
((h != NULL) && (t != NULL)), return;);
/* proceed to last pbuf of chain */
for(p = h; p->next != NULL; p = p->next){
/*使 p = h，当p的next指针(即h的next指针)不是空的时候，即后面还有pbuf*/
/* add total length of second chain to all totals of first chain */
/*第二个链的总长度加到第一个链上的总和*/
p->tot_len += t->tot_len;
}
/* { p is last pbuf of first h chain, p->next == NULL } */
/* p是第一个 h链的末尾pbuf，p->next为空，也就是p的后面再也没有pbuf*/
LWIP_ASSERT("p->tot_len == p->len (of last pbuf in chain)", p->tot_len == p->len);
/* 我还没找到LWIP_ASSERT()是什么，感觉是在声明注释说的意思。有知道的请告诉我 */
LWIP_ASSERT("p->next == NULL", p->next == NULL);
/* add total length of second chain to last pbuf total of first chain */
p->tot_len += t->tot_len;
/* chain last pbuf of head (p) with first of tail (t) */
/* 连接后一个pbuf的头(即 p的头)和第一个的尾部(即 t的尾部)*/
p->next = t;
/* p->next now references t, but the caller will drop its reference to t,
* so netto there is no change to the reference count of t.
* p的next指针现在引用t，但是呼叫方将放弃引用t。所以netto没有改变t的引用计数
*/
}3、pbuf_chain()
/**
* Chain two pbufs (or pbuf chains) together.
* 连接两个pbufs(或者pbufs链表)在一起
* The caller MUST call pbuf_free(t) once it has stopped
* using it. Use pbuf_cat() instead if you no longer use t.
* 呼叫方必须调用pbuf_free（t）一旦终止使用它。使用pbuf_cat()而不是你不在使用t
* @param h head pbuf (chain)
* @param t tail pbuf (chain)
* @note The pbufs MUST belong to the same packet.
* @note MAY NOT be called on a packet queue.
*
* The ->tot_len fields of all pbufs of the head chain are adjusted.
* The ->next field of the last pbuf of the head chain is adjusted.
* The ->ref field of the first pbuf of the tail chain is adjusted.
*
*/
void
pbuf_chain(struct pbuf *h, struct pbuf *t)
{
pbuf_cat(h, t); //将 h和 t进行连接
/* t is now referenced by h t现在被h引用*/
pbuf_ref(t); /* 将t的ref加1，也就是t被引用了一次*/
LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_chain: %p references %p\n", (void *)h, (void *)t));
}4、pbuf的释放函数pbuf_free()
pbuf的申请主要是通过两种方式实现的，即内存池分配与内存堆分配而得到的，所以pbuf的释放也要按照两种方式进行。
在删除一个pbuf结构之前，首先要检查这个pbuf是那种类型的，根据类型的不同，使用不同的内存释放函数进程删除，即内存堆释放函数或者内存池释放函数。PBUF_POOL、PBUF_ROM、PBUF_REF是由内存池分配的，需要调用memp_free()函数进行删除，而PBUF_RAM是由内存堆分配的，需要调用mem_free()函数进行删除。
/**
* Dereference a pbuf chain or queue and deallocate any no-longer-used
* pbufs at the head of this chain or queue.
*
* Decrements the pbuf reference count. If it reaches zero, the pbuf is
* deallocated.
*
* For a pbuf chain, this is repeated for each pbuf in the chain,
* up to the first pbuf which has a non-zero reference count after
* decrementing. So, when all reference counts are one, the whole
* chain is free'd.
*
* @param p The pbuf (chain) to be dereferenced.
*
* @return the number of pbufs that were de-allocated
* from the head of the chain.
*
* @note MUST NOT be called on a packet queue (Not verified to work yet).
* @note the reference counter of a pbuf equals the number of pointers
* that refer to the pbuf (or into the pbuf).
*
* @internal examples:
*
* Assuming existing chains a->b->c with the following reference
* counts, calling pbuf_free(a) results in:
*
* 1->2->3 becomes ...1->3
* 3->3->3 becomes 2->3->3
* 1->1->2 becomes ......1
* 2->1->1 becomes 1->1->1
* 1->1->1 becomes .......
*
*/
u8_t
pbuf_free(struct pbuf *p)
{
u16_t type;
struct pbuf *q;
u8_t count;
if (p == NULL) {
LWIP_ASSERT("p != NULL", p != NULL);
/* if assertions are disabled, proceed with debug output */
LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_LEVEL_SERIOUS,
("pbuf_free(p == NULL) was called.\n"));
return 0;
}
LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_free(%p)\n", (void *)p));
PERF_START;
LWIP_ASSERT("pbuf_free: sane type",
p->type == PBUF_RAM || p->type == PBUF_ROM ||
p->type == PBUF_REF || p->type == PBUF_POOL);
count = 0;
/* de-allocate all consecutive pbufs from the head of the chain that
* obtain a zero reference count after decrementing*/
while (p != NULL) {
u16_t ref;
SYS_ARCH_DECL_PROTECT(old_level);
/* Since decrementing ref cannot be guaranteed to be a single machine operation
* we must protect it. We put the new ref into a local variable to prevent
* further protection. */
SYS_ARCH_PROTECT(old_level);
/* all pbufs in a chain are referenced at least once */
LWIP_ASSERT("pbuf_free: p->ref > 0", p->ref > 0);
/* decrease reference count (number of pointers to pbuf) */
ref = --(p->ref);
SYS_ARCH_UNPROTECT(old_level);
/* this pbuf is no longer referenced to? */
if (ref == 0) {
/* remember next pbuf in chain for next iteration */
q = p->next;
LWIP_DEBUGF( PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_free: deallocating %p\n", (void *)p));
type = p->type;
#if LWIP_SUPPORT_CUSTOM_PBUF
/* is this a custom pbuf? */
if ((p->flags & PBUF_FLAG_IS_CUSTOM) != 0) {
struct pbuf_custom *pc = (struct pbuf_custom*)p;
LWIP_ASSERT("pc->custom_free_function != NULL", pc->custom_free_function != NULL);
pc->custom_free_function(p);
} else
#endif /* LWIP_SUPPORT_CUSTOM_PBUF */
{
/* is this a pbuf from the pool? */
if (type == PBUF_POOL) {
/*PBUF_POOL是MEMP_PBUF_POOL类型的，由内存池分配，调用memp_free()*/
memp_free(MEMP_PBUF_POOL, p);
/* is this a ROM or RAM referencing pbuf? */
} else if (type == PBUF_ROM || type == PBUF_REF) {
/*PBUF_ROM和PBUF_REF是MEMP_PBUF类型的，由内存池分配，调用memp_free()*/
memp_free(MEMP_PBUF, p);
/* type == PBUF_RAM */
} else {
/*PBUF_RAM是由内存堆分配的，调用mem_free进行释放*/
mem_free(p);
}
}
count++;
/* proceed to next pbuf */
p = q;
/* p->ref > 0, this pbuf is still referenced to */
/* (and so the remaining pbufs in chain as well) */
} else {
LWIP_DEBUGF( PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_free: %p has ref %"U16_F", ending here.\n", (void *)p, ref));
/* stop walking through the chain */
p = NULL;
}
}
PERF_STOP("pbuf_free");
/* return number of de-allocated pbufs */
return count;
}
