This patch optimizes io_submit_one to call aio_run_iocb() directly if
ctx->run_list is empty.  When the list is empty, the operation of adding to
the list, then call to __aio_run_iocbs() is unnecessary because these
operations are done in one atomic step.  ctx->run_list always has only one
element in this case.  This optimization speeds up industry standard db
transaction processing benchmark by 0.2%.
Signed-off-by: NKen Chen <kenneth.w.chen@intel.com>
Cc: Benjamin LaHaise <bcrl@kvack.org>
Cc: Suparna Bhattacharya <suparna@in.ibm.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Clean up code that was previously used for debug purpose.  Remove aio_run,
aio_wakeups, iocb->ki_queued and iocb->ki_kicked.  Also clean up unused
variable count in __aio_run_iocbs() and debug code in read_events().
Signed-off-by: NKen Chen <kenneth.w.chen@intel.com>
Cc: Benjamin LaHaise <bcrl@kvack.org>
Cc: Suparna Bhattacharya <suparna@in.ibm.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Since the tail pointer in aio_ring structure never wrap ring size more than
once, so a simple compare is sufficient to wrap the index around.  This avoid
a more expensive mod operation.
Signed-off-by: NKen Chen <kenneth.w.chen@intel.com>
Cc: Benjamin LaHaise <bcrl@kvack.org>
Cc: Suparna Bhattacharya <suparna@in.ibm.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

This patch removes superfluous kiocb member initialization in the AIO
allocation and deallocation path.  For example, in really_put_req(),
right before kiocb is returned to slab, 5 variables are reset to NULL.
The same variables will be initialized at the kiocb allocation time,
so why bother reset them knowing that they will be set to valid data
at alloc time?  Another example: ki_retry is initialized in __aio_get_req,
but is initialized again in io_submit_one.
Signed-off-by: NKen Chen <kenneth.w.chen@intel.com>
Cc: Benjamin LaHaise <bcrl@kvack.org>
Cc: Suparna Bhattacharya <suparna@in.ibm.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

This patch makes some needlessly global code static.
Signed-off-by: NAdrian Bunk <bunk@stusta.de>
Acked-by: NBenjamin LaHaise <bcrl@kvack.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!