提交 5a2dd72a 编写于 作者: I Ingo Molnar

Merge branch 'linus' into irq/genirq

要显示的变更太多。

To preserve performance only 1000 of 1000+ files are displayed.
......@@ -32,6 +32,7 @@ Christoph Hellwig <hch@lst.de>
Corey Minyard <minyard@acm.org>
David Brownell <david-b@pacbell.net>
David Woodhouse <dwmw2@shinybook.infradead.org>
Dmitry Eremin-Solenikov <dbaryshkov@gmail.com>
Domen Puncer <domen@coderock.org>
Douglas Gilbert <dougg@torque.net>
Ed L. Cashin <ecashin@coraid.com>
......
......@@ -369,10 +369,10 @@ P: 1024/8462A731 4C 55 86 34 44 59 A7 99 2B 97 88 4A 88 9A 0D 97
D: sun4 port, Sparc hacker
N: Hugh Blemings
E: hugh@misc.nu
W: http://misc.nu/hugh/
D: Author and maintainer of the Keyspan USB to Serial drivers
S: Po Box 234
E: hugh@blemings.org
W: http://blemings.org/hugh
D: Original author of the Keyspan USB to serial drivers, random PowerPC hacker
S: PO Box 234
S: Belconnen ACT 2616
S: Australia
......@@ -464,6 +464,11 @@ S: 1200 Goldenrod Dr.
S: Nampa, Idaho 83686
S: USA
N: Dirk J. Brandewie
E: dirk.j.brandewie@intel.com
E: linux-wimax@intel.com
D: Intel Wireless WiMAX Connection 2400 SDIO driver
N: Derrick J. Brashear
E: shadow@dementia.org
W: http://www.dementia.org/~shadow
......@@ -1681,7 +1686,7 @@ E: ajoshi@shell.unixbox.com
D: fbdev hacking
N: Jesper Juhl
E: jesper.juhl@gmail.com
E: jj@chaosbits.net
D: Various fixes, cleanups and minor features all over the tree.
D: Wrote initial version of the hdaps driver (since passed on to others).
S: Lemnosvej 1, 3.tv
......@@ -2119,6 +2124,11 @@ N: H.J. Lu
E: hjl@gnu.ai.mit.edu
D: GCC + libraries hacker
N: Yanir Lubetkin
E: yanirx.lubatkin@intel.com
E: linux-wimax@intel.com
D: Intel Wireless WiMAX Connection 2400 driver
N: Michal Ludvig
E: michal@logix.cz
E: michal.ludvig@asterisk.co.nz
......@@ -2693,6 +2703,13 @@ S: RR #5, 497 Pole Line Road
S: Thunder Bay, Ontario
S: CANADA P7C 5M9
N: Inaky Perez-Gonzalez
E: inaky.perez-gonzalez@intel.com
E: linux-wimax@intel.com
E: inakypg@yahoo.com
D: WiMAX stack
D: Intel Wireless WiMAX Connection 2400 driver
N: Yuri Per
E: yuri@pts.mipt.ru
D: Some smbfs fixes
......@@ -3769,14 +3786,11 @@ S: The Netherlands
N: David Woodhouse
E: dwmw2@infradead.org
D: ARCnet stuff, Applicom board driver, SO_BINDTODEVICE,
D: some Alpha platform porting from 2.0, Memory Technology Devices,
D: Acquire watchdog timer, PC speaker driver maintenance,
D: JFFS2 file system, Memory Technology Device subsystem,
D: various other stuff that annoyed me by not working.
S: c/o Red Hat Engineering
S: Rustat House
S: 60 Clifton Road
S: Cambridge. CB1 7EG
S: c/o Intel Corporation
S: Pipers Way
S: Swindon. SN3 1RJ
S: England
N: Chris Wright
......
......@@ -3,8 +3,9 @@ Date: April 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
state. This holds the regulator output state.
Some regulator directories will contain a field called
state. This reports the regulator enable status, for
regulators which can report that value.
This will be one of the following strings:
......@@ -18,7 +19,8 @@ Description:
'disabled' means the regulator output is OFF and is not
supplying power to the system..
'unknown' means software cannot determine the state.
'unknown' means software cannot determine the state, or
the reported state is invalid.
NOTE: this field can be used in conjunction with microvolts
and microamps to determine regulator output levels.
......@@ -53,9 +55,10 @@ Date: April 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
microvolts. This holds the regulator output voltage setting
measured in microvolts (i.e. E-6 Volts).
measured in microvolts (i.e. E-6 Volts), for regulators
which can report that voltage.
NOTE: This value should not be used to determine the regulator
output voltage level as this value is the same regardless of
......@@ -67,9 +70,10 @@ Date: April 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
microamps. This holds the regulator output current limit
setting measured in microamps (i.e. E-6 Amps).
setting measured in microamps (i.e. E-6 Amps), for regulators
which can report that current.
NOTE: This value should not be used to determine the regulator
output current level as this value is the same regardless of
......@@ -81,8 +85,9 @@ Date: April 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
opmode. This holds the regulator operating mode setting.
Some regulator directories will contain a field called
opmode. This holds the current regulator operating mode,
for regulators which can report it.
The opmode value can be one of the following strings:
......@@ -92,7 +97,7 @@ Description:
'standby'
'unknown'
The modes are described in include/linux/regulator/regulator.h
The modes are described in include/linux/regulator/consumer.h
NOTE: This value should not be used to determine the regulator
output operating mode as this value is the same regardless of
......@@ -104,9 +109,10 @@ Date: April 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
min_microvolts. This holds the minimum safe working regulator
output voltage setting for this domain measured in microvolts.
output voltage setting for this domain measured in microvolts,
for regulators which support voltage constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no min microvolts constraint defined by
......@@ -118,9 +124,10 @@ Date: April 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
max_microvolts. This holds the maximum safe working regulator
output voltage setting for this domain measured in microvolts.
output voltage setting for this domain measured in microvolts,
for regulators which support voltage constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no max microvolts constraint defined by
......@@ -132,10 +139,10 @@ Date: April 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
min_microamps. This holds the minimum safe working regulator
output current limit setting for this domain measured in
microamps.
microamps, for regulators which support current constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no min microamps constraint defined by
......@@ -147,10 +154,10 @@ Date: April 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
max_microamps. This holds the maximum safe working regulator
output current limit setting for this domain measured in
microamps.
microamps, for regulators which support current constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no max microamps constraint defined by
......@@ -185,7 +192,7 @@ Date: April 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
requested_microamps. This holds the total requested load
current in microamps for this regulator from all its consumer
devices.
......@@ -204,125 +211,102 @@ Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
suspend_mem_microvolts. This holds the regulator output
voltage setting for this domain measured in microvolts when
the system is suspended to memory.
NOTE: this will return the string 'not defined' if
the power domain has no suspend to memory voltage defined by
platform code.
the system is suspended to memory, for voltage regulators
implementing suspend voltage configuration constraints.
What: /sys/class/regulator/.../suspend_disk_microvolts
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
suspend_disk_microvolts. This holds the regulator output
voltage setting for this domain measured in microvolts when
the system is suspended to disk.
NOTE: this will return the string 'not defined' if
the power domain has no suspend to disk voltage defined by
platform code.
the system is suspended to disk, for voltage regulators
implementing suspend voltage configuration constraints.
What: /sys/class/regulator/.../suspend_standby_microvolts
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
suspend_standby_microvolts. This holds the regulator output
voltage setting for this domain measured in microvolts when
the system is suspended to standby.
NOTE: this will return the string 'not defined' if
the power domain has no suspend to standby voltage defined by
platform code.
the system is suspended to standby, for voltage regulators
implementing suspend voltage configuration constraints.
What: /sys/class/regulator/.../suspend_mem_mode
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
suspend_mem_mode. This holds the regulator operating mode
setting for this domain when the system is suspended to
memory.
NOTE: this will return the string 'not defined' if
the power domain has no suspend to memory mode defined by
platform code.
memory, for regulators implementing suspend mode
configuration constraints.
What: /sys/class/regulator/.../suspend_disk_mode
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
suspend_disk_mode. This holds the regulator operating mode
setting for this domain when the system is suspended to disk.
NOTE: this will return the string 'not defined' if
the power domain has no suspend to disk mode defined by
platform code.
setting for this domain when the system is suspended to disk,
for regulators implementing suspend mode configuration
constraints.
What: /sys/class/regulator/.../suspend_standby_mode
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
suspend_standby_mode. This holds the regulator operating mode
setting for this domain when the system is suspended to
standby.
NOTE: this will return the string 'not defined' if
the power domain has no suspend to standby mode defined by
platform code.
standby, for regulators implementing suspend mode
configuration constraints.
What: /sys/class/regulator/.../suspend_mem_state
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
suspend_mem_state. This holds the regulator operating state
when suspended to memory.
This will be one of the following strings:
when suspended to memory, for regulators implementing suspend
configuration constraints.
'enabled'
'disabled'
'not defined'
This will be one of the same strings reported by
the "state" attribute.
What: /sys/class/regulator/.../suspend_disk_state
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
suspend_disk_state. This holds the regulator operating state
when suspended to disk.
This will be one of the following strings:
when suspended to disk, for regulators implementing
suspend configuration constraints.
'enabled'
'disabled'
'not defined'
This will be one of the same strings reported by
the "state" attribute.
What: /sys/class/regulator/.../suspend_standby_state
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
Each regulator directory will contain a field called
Some regulator directories will contain a field called
suspend_standby_state. This holds the regulator operating
state when suspended to standby.
This will be one of the following strings:
state when suspended to standby, for regulators implementing
suspend configuration constraints.
'enabled'
'disabled'
'not defined'
This will be one of the same strings reported by
the "state" attribute.
......@@ -32,14 +32,16 @@ Contact: linux-usb@vger.kernel.org
Description:
Write:
<channel> [<bpst offset>]
<channel>
to start beaconing on a specific channel, or stop
beaconing if <channel> is -1. Valid channels depends
on the radio controller's supported band groups.
to force a specific channel to be used when beaconing,
or, if <channel> is -1, to prohibit beaconing. If
<channel> is 0, then the default channel selection
algorithm will be used. Valid channels depends on the
radio controller's supported band groups.
<bpst offset> may be used to try and join a specific
beacon group if more than one was found during a scan.
Reading returns the currently active channel, or -1 if
the radio controller is not beaconing.
What: /sys/class/uwb_rc/uwbN/scan
Date: July 2008
......
......@@ -6,7 +6,6 @@ Description:
internal state of the kernel memory blocks. Files could be
added or removed dynamically to represent hot-add/remove
operations.
Users: hotplug memory add/remove tools
https://w3.opensource.ibm.com/projects/powerpc-utils/
......@@ -19,6 +18,56 @@ Description:
This is useful for a user-level agent to determine
identify removable sections of the memory before attempting
potentially expensive hot-remove memory operation
Users: hotplug memory remove tools
https://w3.opensource.ibm.com/projects/powerpc-utils/
What: /sys/devices/system/memory/memoryX/phys_device
Date: September 2008
Contact: Badari Pulavarty <pbadari@us.ibm.com>
Description:
The file /sys/devices/system/memory/memoryX/phys_device
is read-only and is designed to show the name of physical
memory device. Implementation is currently incomplete.
What: /sys/devices/system/memory/memoryX/phys_index
Date: September 2008
Contact: Badari Pulavarty <pbadari@us.ibm.com>
Description:
The file /sys/devices/system/memory/memoryX/phys_index
is read-only and contains the section ID in hexadecimal
which is equivalent to decimal X contained in the
memory section directory name.
What: /sys/devices/system/memory/memoryX/state
Date: September 2008
Contact: Badari Pulavarty <pbadari@us.ibm.com>
Description:
The file /sys/devices/system/memory/memoryX/state
is read-write. When read, it's contents show the
online/offline state of the memory section. When written,
root can toggle the the online/offline state of a removable
memory section (see removable file description above)
using the following commands.
# echo online > /sys/devices/system/memory/memoryX/state
# echo offline > /sys/devices/system/memory/memoryX/state
For example, if /sys/devices/system/memory/memory22/removable
contains a value of 1 and
/sys/devices/system/memory/memory22/state contains the
string "online" the following command can be executed by
by root to offline that section.
# echo offline > /sys/devices/system/memory/memory22/state
Users: hotplug memory remove tools
https://w3.opensource.ibm.com/projects/powerpc-utils/
What: /sys/devices/system/node/nodeX/memoryY
Date: September 2008
Contact: Gary Hade <garyhade@us.ibm.com>
Description:
When CONFIG_NUMA is enabled
/sys/devices/system/node/nodeX/memoryY is a symbolic link that
points to the corresponding /sys/devices/system/memory/memoryY
memory section directory. For example, the following symbolic
link is created for memory section 9 on node0.
/sys/devices/system/node/node0/memory9 -> ../../memory/memory9
......@@ -170,16 +170,15 @@ Returns: 0 if successful and a negative error if not.
u64
dma_get_required_mask(struct device *dev)
After setting the mask with dma_set_mask(), this API returns the
actual mask (within that already set) that the platform actually
requires to operate efficiently. Usually this means the returned mask
This API returns the mask that the platform requires to
operate efficiently. Usually this means the returned mask
is the minimum required to cover all of memory. Examining the
required mask gives drivers with variable descriptor sizes the
opportunity to use smaller descriptors as necessary.
Requesting the required mask does not alter the current mask. If you
wish to take advantage of it, you should issue another dma_set_mask()
call to lower the mask again.
wish to take advantage of it, you should issue a dma_set_mask()
call to set the mask to the value returned.
Part Id - Streaming DMA mappings
......
......@@ -26,7 +26,7 @@ mapped only for the time they are actually used and unmapped after the DMA
transfer.
The following API will work of course even on platforms where no such
hardware exists, see e.g. include/asm-i386/pci.h for how it is implemented on
hardware exists, see e.g. arch/x86/include/asm/pci.h for how it is implemented on
top of the virt_to_bus interface.
First of all, you should make sure
......
......@@ -6,13 +6,13 @@
# To add a new book the only step required is to add the book to the
# list of DOCBOOKS.
DOCBOOKS := wanbook.xml z8530book.xml mcabook.xml \
DOCBOOKS := z8530book.xml mcabook.xml \
kernel-hacking.xml kernel-locking.xml deviceiobook.xml \
procfs-guide.xml writing_usb_driver.xml networking.xml \
kernel-api.xml filesystems.xml lsm.xml usb.xml kgdb.xml \
gadget.xml libata.xml mtdnand.xml librs.xml rapidio.xml \
genericirq.xml s390-drivers.xml uio-howto.xml scsi.xml \
mac80211.xml debugobjects.xml sh.xml
mac80211.xml debugobjects.xml sh.xml regulator.xml
###
# The build process is as follows (targets):
......
......@@ -74,6 +74,14 @@
!Enet/sunrpc/rpcb_clnt.c
!Enet/sunrpc/clnt.c
</sect1>
<sect1><title>WiMAX</title>
!Enet/wimax/op-msg.c
!Enet/wimax/op-reset.c
!Enet/wimax/op-rfkill.c
!Enet/wimax/stack.c
!Iinclude/net/wimax.h
!Iinclude/linux/wimax.h
</sect1>
</chapter>
<chapter id="netdev">
......@@ -98,9 +106,6 @@
X!Enet/core/wireless.c
</sect1>
-->
<sect1><title>Synchronous PPP</title>
!Edrivers/net/wan/syncppp.c
</sect1>
</chapter>
</book>
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
"http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" []>
<book id="regulator-api">
<bookinfo>
<title>Voltage and current regulator API</title>
<authorgroup>
<author>
<firstname>Liam</firstname>
<surname>Girdwood</surname>
<affiliation>
<address>
<email>lrg@slimlogic.co.uk</email>
</address>
</affiliation>
</author>
<author>
<firstname>Mark</firstname>
<surname>Brown</surname>
<affiliation>
<orgname>Wolfson Microelectronics</orgname>
<address>
<email>broonie@opensource.wolfsonmicro.com</email>
</address>
</affiliation>
</author>
</authorgroup>
<copyright>
<year>2007-2008</year>
<holder>Wolfson Microelectronics</holder>
</copyright>
<copyright>
<year>2008</year>
<holder>Liam Girdwood</holder>
</copyright>
<legalnotice>
<para>
This documentation is free software; you can redistribute
it and/or modify it under the terms of the GNU General Public
License version 2 as published by the Free Software Foundation.
</para>
<para>
This program is distributed in the hope that it will be
useful, but WITHOUT ANY WARRANTY; without even the implied
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details.
</para>
<para>
You should have received a copy of the GNU General Public
License along with this program; if not, write to the Free
Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
MA 02111-1307 USA
</para>
<para>
For more details see the file COPYING in the source
distribution of Linux.
</para>
</legalnotice>
</bookinfo>
<toc></toc>
<chapter id="intro">
<title>Introduction</title>
<para>
This framework is designed to provide a standard kernel
interface to control voltage and current regulators.
</para>
<para>
The intention is to allow systems to dynamically control
regulator power output in order to save power and prolong
battery life. This applies to both voltage regulators (where
voltage output is controllable) and current sinks (where current
limit is controllable).
</para>
<para>
Note that additional (and currently more complete) documentation
is available in the Linux kernel source under
<filename>Documentation/power/regulator</filename>.
</para>
<sect1 id="glossary">
<title>Glossary</title>
<para>
The regulator API uses a number of terms which may not be
familiar:
</para>
<glossary>
<glossentry>
<glossterm>Regulator</glossterm>
<glossdef>
<para>
Electronic device that supplies power to other devices. Most
regulators can enable and disable their output and some can also
control their output voltage or current.
</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Consumer</glossterm>
<glossdef>
<para>
Electronic device which consumes power provided by a regulator.
These may either be static, requiring only a fixed supply, or
dynamic, requiring active management of the regulator at
runtime.
</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Power Domain</glossterm>
<glossdef>
<para>
The electronic circuit supplied by a given regulator, including
the regulator and all consumer devices. The configuration of
the regulator is shared between all the components in the
circuit.
</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Power Management Integrated Circuit</glossterm>
<acronym>PMIC</acronym>
<glossdef>
<para>
An IC which contains numerous regulators and often also other
subsystems. In an embedded system the primary PMIC is often
equivalent to a combination of the PSU and southbridge in a
desktop system.
</para>
</glossdef>
</glossentry>
</glossary>
</sect1>
</chapter>
<chapter id="consumer">
<title>Consumer driver interface</title>
<para>
This offers a similar API to the kernel clock framework.
Consumer drivers use <link
linkend='API-regulator-get'>get</link> and <link
linkend='API-regulator-put'>put</link> operations to acquire and
release regulators. Functions are
provided to <link linkend='API-regulator-enable'>enable</link>
and <link linkend='API-regulator-disable'>disable</link> the
reguator and to get and set the runtime parameters of the
regulator.
</para>
<para>
When requesting regulators consumers use symbolic names for their
supplies, such as "Vcc", which are mapped into actual regulator
devices by the machine interface.
</para>
<para>
A stub version of this API is provided when the regulator
framework is not in use in order to minimise the need to use
ifdefs.
</para>
<sect1 id="consumer-enable">
<title>Enabling and disabling</title>
<para>
The regulator API provides reference counted enabling and
disabling of regulators. Consumer devices use the <function><link
linkend='API-regulator-enable'>regulator_enable</link></function>
and <function><link
linkend='API-regulator-disable'>regulator_disable</link>
</function> functions to enable and disable regulators. Calls
to the two functions must be balanced.
</para>
<para>
Note that since multiple consumers may be using a regulator and
machine constraints may not allow the regulator to be disabled
there is no guarantee that calling
<function>regulator_disable</function> will actually cause the
supply provided by the regulator to be disabled. Consumer
drivers should assume that the regulator may be enabled at all
times.
</para>
</sect1>
<sect1 id="consumer-config">
<title>Configuration</title>
<para>
Some consumer devices may need to be able to dynamically
configure their supplies. For example, MMC drivers may need to
select the correct operating voltage for their cards. This may
be done while the regulator is enabled or disabled.
</para>
<para>
The <function><link
linkend='API-regulator-set-voltage'>regulator_set_voltage</link>
</function> and <function><link
linkend='API-regulator-set-current-limit'
>regulator_set_current_limit</link>
</function> functions provide the primary interface for this.
Both take ranges of voltages and currents, supporting drivers
that do not require a specific value (eg, CPU frequency scaling
normally permits the CPU to use a wider range of supply
voltages at lower frequencies but does not require that the
supply voltage be lowered). Where an exact value is required
both minimum and maximum values should be identical.
</para>
</sect1>
<sect1 id="consumer-callback">
<title>Callbacks</title>
<para>
Callbacks may also be <link
linkend='API-regulator-register-notifier'>registered</link>
for events such as regulation failures.
</para>
</sect1>
</chapter>
<chapter id="driver">
<title>Regulator driver interface</title>
<para>
Drivers for regulator chips <link
linkend='API-regulator-register'>register</link> the regulators
with the regulator core, providing operations structures to the
core. A <link
linkend='API-regulator-notifier-call-chain'>notifier</link> interface
allows error conditions to be reported to the core.
</para>
<para>
Registration should be triggered by explicit setup done by the
platform, supplying a <link
linkend='API-struct-regulator-init-data'>struct
regulator_init_data</link> for the regulator containing
<link linkend='machine-constraint'>constraint</link> and
<link linkend='machine-supply'>supply</link> information.
</para>
</chapter>
<chapter id="machine">
<title>Machine interface</title>
<para>
This interface provides a way to define how regulators are
connected to consumers on a given system and what the valid
operating parameters are for the system.
</para>
<sect1 id="machine-supply">
<title>Supplies</title>
<para>
Regulator supplies are specified using <link
linkend='API-struct-regulator-consumer-supply'>struct
regulator_consumer_supply</link>. This is done at
<link linkend='driver'>driver registration
time</link> as part of the machine constraints.
</para>
</sect1>
<sect1 id="machine-constraint">
<title>Constraints</title>
<para>
As well as definining the connections the machine interface
also provides constraints definining the operations that
clients are allowed to perform and the parameters that may be
set. This is required since generally regulator devices will
offer more flexibility than it is safe to use on a given
system, for example supporting higher supply voltages than the
consumers are rated for.
</para>
<para>
This is done at <link linkend='driver'>driver
registration time</link> by providing a <link
linkend='API-struct-regulation-constraints'>struct
regulation_constraints</link>.
</para>
<para>
The constraints may also specify an initial configuration for the
regulator in the constraints, which is particularly useful for
use with static consumers.
</para>
</sect1>
</chapter>
<chapter id="api">
<title>API reference</title>
<para>
Due to limitations of the kernel documentation framework and the
existing layout of the source code the entire regulator API is
documented here.
</para>
!Iinclude/linux/regulator/consumer.h
!Iinclude/linux/regulator/machine.h
!Iinclude/linux/regulator/driver.h
!Edrivers/regulator/core.c
</chapter>
</book>
......@@ -41,6 +41,12 @@ GPL version 2.
</abstract>
<revhistory>
<revision>
<revnumber>0.6</revnumber>
<date>2008-12-05</date>
<authorinitials>hjk</authorinitials>
<revremark>Added description of portio sysfs attributes.</revremark>
</revision>
<revision>
<revnumber>0.5</revnumber>
<date>2008-05-22</date>
......@@ -318,6 +324,54 @@ interested in translating it, please email me
offset = N * getpagesize();
</programlisting>
<para>
Sometimes there is hardware with memory-like regions that can not be
mapped with the technique described here, but there are still ways to
access them from userspace. The most common example are x86 ioports.
On x86 systems, userspace can access these ioports using
<function>ioperm()</function>, <function>iopl()</function>,
<function>inb()</function>, <function>outb()</function>, and similar
functions.
</para>
<para>
Since these ioport regions can not be mapped, they will not appear under
<filename>/sys/class/uio/uioX/maps/</filename> like the normal memory
described above. Without information about the port regions a hardware
has to offer, it becomes difficult for the userspace part of the
driver to find out which ports belong to which UIO device.
</para>
<para>
To address this situation, the new directory
<filename>/sys/class/uio/uioX/portio/</filename> was added. It only
exists if the driver wants to pass information about one or more port
regions to userspace. If that is the case, subdirectories named
<filename>port0</filename>, <filename>port1</filename>, and so on,
will appear underneath
<filename>/sys/class/uio/uioX/portio/</filename>.
</para>
<para>
Each <filename>portX/</filename> directory contains three read-only
files that show start, size, and type of the port region:
</para>
<itemizedlist>
<listitem>
<para>
<filename>start</filename>: The first port of this region.
</para>
</listitem>
<listitem>
<para>
<filename>size</filename>: The number of ports in this region.
</para>
</listitem>
<listitem>
<para>
<filename>porttype</filename>: A string describing the type of port.
</para>
</listitem>
</itemizedlist>
</sect1>
</chapter>
......@@ -339,12 +393,12 @@ offset = N * getpagesize();
<itemizedlist>
<listitem><para>
<varname>char *name</varname>: Required. The name of your driver as
<varname>const char *name</varname>: Required. The name of your driver as
it will appear in sysfs. I recommend using the name of your module for this.
</para></listitem>
<listitem><para>
<varname>char *version</varname>: Required. This string appears in
<varname>const char *version</varname>: Required. This string appears in
<filename>/sys/class/uio/uioX/version</filename>.
</para></listitem>
......@@ -355,6 +409,13 @@ mapping you need to fill one of the <varname>uio_mem</varname> structures.
See the description below for details.
</para></listitem>
<listitem><para>
<varname>struct uio_port port[ MAX_UIO_PORTS_REGIONS ]</varname>: Required
if you want to pass information about ioports to userspace. For each port
region you need to fill one of the <varname>uio_port</varname> structures.
See the description below for details.
</para></listitem>
<listitem><para>
<varname>long irq</varname>: Required. If your hardware generates an
interrupt, it's your modules task to determine the irq number during
......@@ -448,6 +509,42 @@ Please do not touch the <varname>kobj</varname> element of
<varname>struct uio_mem</varname>! It is used by the UIO framework
to set up sysfs files for this mapping. Simply leave it alone.
</para>
<para>
Sometimes, your device can have one or more port regions which can not be
mapped to userspace. But if there are other possibilities for userspace to
access these ports, it makes sense to make information about the ports
available in sysfs. For each region, you have to set up a
<varname>struct uio_port</varname> in the <varname>port[]</varname> array.
Here's a description of the fields of <varname>struct uio_port</varname>:
</para>
<itemizedlist>
<listitem><para>
<varname>char *porttype</varname>: Required. Set this to one of the predefined
constants. Use <varname>UIO_PORT_X86</varname> for the ioports found in x86
architectures.
</para></listitem>
<listitem><para>
<varname>unsigned long start</varname>: Required if the port region is used.
Fill in the number of the first port of this region.
</para></listitem>
<listitem><para>
<varname>unsigned long size</varname>: Fill in the number of ports in this
region. If <varname>size</varname> is zero, the region is considered unused.
Note that you <emphasis>must</emphasis> initialize <varname>size</varname>
with zero for all unused regions.
</para></listitem>
</itemizedlist>
<para>
Please do not touch the <varname>portio</varname> element of
<varname>struct uio_port</varname>! It is used internally by the UIO
framework to set up sysfs files for this region. Simply leave it alone.
</para>
</sect1>
<sect1 id="adding_irq_handler">
......
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
"http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" []>
<book id="WANGuide">
<bookinfo>
<title>Synchronous PPP and Cisco HDLC Programming Guide</title>
<authorgroup>
<author>
<firstname>Alan</firstname>
<surname>Cox</surname>
<affiliation>
<address>
<email>alan@lxorguk.ukuu.org.uk</email>
</address>
</affiliation>
</author>
</authorgroup>
<copyright>
<year>2000</year>
<holder>Alan Cox</holder>
</copyright>
<legalnotice>
<para>
This documentation is free software; you can redistribute
it and/or modify it under the terms of the GNU General Public
License as published by the Free Software Foundation; either
version 2 of the License, or (at your option) any later
version.
</para>
<para>
This program is distributed in the hope that it will be
useful, but WITHOUT ANY WARRANTY; without even the implied
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details.
</para>
<para>
You should have received a copy of the GNU General Public
License along with this program; if not, write to the Free
Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
MA 02111-1307 USA
</para>
<para>
For more details see the file COPYING in the source
distribution of Linux.
</para>
</legalnotice>
</bookinfo>
<toc></toc>
<chapter id="intro">
<title>Introduction</title>
<para>
The syncppp drivers in Linux provide a fairly complete
implementation of Cisco HDLC and a minimal implementation of
PPP. The longer term goal is to switch the PPP layer to the
generic PPP interface that is new in Linux 2.3.x. The API should
remain unchanged when this is done, but support will then be
available for IPX, compression and other PPP features
</para>
</chapter>
<chapter id="bugs">
<title>Known Bugs And Assumptions</title>
<para>
<variablelist>
<varlistentry><term>PPP is minimal</term>
<listitem>
<para>
The current PPP implementation is very basic, although sufficient
for most wan usages.
</para>
</listitem></varlistentry>
<varlistentry><term>Cisco HDLC Quirks</term>
<listitem>
<para>
Currently we do not end all packets with the correct Cisco multicast
or unicast flags. Nothing appears to mind too much but this should
be corrected.
</para>
</listitem></varlistentry>
</variablelist>
</para>
</chapter>
<chapter id="pubfunctions">
<title>Public Functions Provided</title>
!Edrivers/net/wan/syncppp.c
</chapter>
</book>
......@@ -294,7 +294,8 @@ NOTE: pci_enable_device() can fail! Check the return value.
pci_set_master() will enable DMA by setting the bus master bit
in the PCI_COMMAND register. It also fixes the latency timer value if
it's set to something bogus by the BIOS.
it's set to something bogus by the BIOS. pci_clear_master() will
disable DMA by clearing the bus master bit.
If the PCI device can use the PCI Memory-Write-Invalidate transaction,
call pci_set_mwi(). This enables the PCI_COMMAND bit for Mem-Wr-Inval
......
......@@ -12,10 +12,14 @@ rcuref.txt
- Reference-count design for elements of lists/arrays protected by RCU
rcu.txt
- RCU Concepts
rcubarrier.txt
- Unloading modules that use RCU callbacks
RTFP.txt
- List of RCU papers (bibliography) going back to 1980.
torture.txt
- RCU Torture Test Operation (CONFIG_RCU_TORTURE_TEST)
trace.txt
- CONFIG_RCU_TRACE debugfs files and formats
UP.txt
- RCU on Uniprocessor Systems
whatisRCU.txt
......
RCU and Unloadable Modules
[Originally published in LWN Jan. 14, 2007: http://lwn.net/Articles/217484/]
RCU (read-copy update) is a synchronization mechanism that can be thought
of as a replacement for read-writer locking (among other things), but with
very low-overhead readers that are immune to deadlock, priority inversion,
and unbounded latency. RCU read-side critical sections are delimited
by rcu_read_lock() and rcu_read_unlock(), which, in non-CONFIG_PREEMPT
kernels, generate no code whatsoever.
This means that RCU writers are unaware of the presence of concurrent
readers, so that RCU updates to shared data must be undertaken quite
carefully, leaving an old version of the data structure in place until all
pre-existing readers have finished. These old versions are needed because
such readers might hold a reference to them. RCU updates can therefore be
rather expensive, and RCU is thus best suited for read-mostly situations.
How can an RCU writer possibly determine when all readers are finished,
given that readers might well leave absolutely no trace of their
presence? There is a synchronize_rcu() primitive that blocks until all
pre-existing readers have completed. An updater wishing to delete an
element p from a linked list might do the following, while holding an
appropriate lock, of course:
list_del_rcu(p);
synchronize_rcu();
kfree(p);
But the above code cannot be used in IRQ context -- the call_rcu()
primitive must be used instead. This primitive takes a pointer to an
rcu_head struct placed within the RCU-protected data structure and
another pointer to a function that may be invoked later to free that
structure. Code to delete an element p from the linked list from IRQ
context might then be as follows:
list_del_rcu(p);
call_rcu(&p->rcu, p_callback);
Since call_rcu() never blocks, this code can safely be used from within
IRQ context. The function p_callback() might be defined as follows:
static void p_callback(struct rcu_head *rp)
{
struct pstruct *p = container_of(rp, struct pstruct, rcu);
kfree(p);
}
Unloading Modules That Use call_rcu()
But what if p_callback is defined in an unloadable module?
If we unload the module while some RCU callbacks are pending,
the CPUs executing these callbacks are going to be severely
disappointed when they are later invoked, as fancifully depicted at
http://lwn.net/images/ns/kernel/rcu-drop.jpg.
We could try placing a synchronize_rcu() in the module-exit code path,
but this is not sufficient. Although synchronize_rcu() does wait for a
grace period to elapse, it does not wait for the callbacks to complete.
One might be tempted to try several back-to-back synchronize_rcu()
calls, but this is still not guaranteed to work. If there is a very
heavy RCU-callback load, then some of the callbacks might be deferred
in order to allow other processing to proceed. Such deferral is required
in realtime kernels in order to avoid excessive scheduling latencies.
rcu_barrier()
We instead need the rcu_barrier() primitive. This primitive is similar
to synchronize_rcu(), but instead of waiting solely for a grace
period to elapse, it also waits for all outstanding RCU callbacks to
complete. Pseudo-code using rcu_barrier() is as follows:
1. Prevent any new RCU callbacks from being posted.
2. Execute rcu_barrier().
3. Allow the module to be unloaded.
Quick Quiz #1: Why is there no srcu_barrier()?
The rcutorture module makes use of rcu_barrier in its exit function
as follows:
1 static void
2 rcu_torture_cleanup(void)
3 {
4 int i;
5
6 fullstop = 1;
7 if (shuffler_task != NULL) {
8 VERBOSE_PRINTK_STRING("Stopping rcu_torture_shuffle task");
9 kthread_stop(shuffler_task);
10 }
11 shuffler_task = NULL;
12
13 if (writer_task != NULL) {
14 VERBOSE_PRINTK_STRING("Stopping rcu_torture_writer task");
15 kthread_stop(writer_task);
16 }
17 writer_task = NULL;
18
19 if (reader_tasks != NULL) {
20 for (i = 0; i < nrealreaders; i++) {
21 if (reader_tasks[i] != NULL) {
22 VERBOSE_PRINTK_STRING(
23 "Stopping rcu_torture_reader task");
24 kthread_stop(reader_tasks[i]);
25 }
26 reader_tasks[i] = NULL;
27 }
28 kfree(reader_tasks);
29 reader_tasks = NULL;
30 }
31 rcu_torture_current = NULL;
32
33 if (fakewriter_tasks != NULL) {
34 for (i = 0; i < nfakewriters; i++) {
35 if (fakewriter_tasks[i] != NULL) {
36 VERBOSE_PRINTK_STRING(
37 "Stopping rcu_torture_fakewriter task");
38 kthread_stop(fakewriter_tasks[i]);
39 }
40 fakewriter_tasks[i] = NULL;
41 }
42 kfree(fakewriter_tasks);
43 fakewriter_tasks = NULL;
44 }
45
46 if (stats_task != NULL) {
47 VERBOSE_PRINTK_STRING("Stopping rcu_torture_stats task");
48 kthread_stop(stats_task);
49 }
50 stats_task = NULL;
51
52 /* Wait for all RCU callbacks to fire. */
53 rcu_barrier();
54
55 rcu_torture_stats_print(); /* -After- the stats thread is stopped! */
56
57 if (cur_ops->cleanup != NULL)
58 cur_ops->cleanup();
59 if (atomic_read(&n_rcu_torture_error))
60 rcu_torture_print_module_parms("End of test: FAILURE");
61 else
62 rcu_torture_print_module_parms("End of test: SUCCESS");
63 }
Line 6 sets a global variable that prevents any RCU callbacks from
re-posting themselves. This will not be necessary in most cases, since
RCU callbacks rarely include calls to call_rcu(). However, the rcutorture
module is an exception to this rule, and therefore needs to set this
global variable.
Lines 7-50 stop all the kernel tasks associated with the rcutorture
module. Therefore, once execution reaches line 53, no more rcutorture
RCU callbacks will be posted. The rcu_barrier() call on line 53 waits
for any pre-existing callbacks to complete.
Then lines 55-62 print status and do operation-specific cleanup, and
then return, permitting the module-unload operation to be completed.
Quick Quiz #2: Is there any other situation where rcu_barrier() might
be required?
Your module might have additional complications. For example, if your
module invokes call_rcu() from timers, you will need to first cancel all
the timers, and only then invoke rcu_barrier() to wait for any remaining
RCU callbacks to complete.
Implementing rcu_barrier()
Dipankar Sarma's implementation of rcu_barrier() makes use of the fact
that RCU callbacks are never reordered once queued on one of the per-CPU
queues. His implementation queues an RCU callback on each of the per-CPU
callback queues, and then waits until they have all started executing, at
which point, all earlier RCU callbacks are guaranteed to have completed.
The original code for rcu_barrier() was as follows:
1 void rcu_barrier(void)
2 {
3 BUG_ON(in_interrupt());
4 /* Take cpucontrol mutex to protect against CPU hotplug */
5 mutex_lock(&rcu_barrier_mutex);
6 init_completion(&rcu_barrier_completion);
7 atomic_set(&rcu_barrier_cpu_count, 0);
8 on_each_cpu(rcu_barrier_func, NULL, 0, 1);
9 wait_for_completion(&rcu_barrier_completion);
10 mutex_unlock(&rcu_barrier_mutex);
11 }
Line 3 verifies that the caller is in process context, and lines 5 and 10
use rcu_barrier_mutex to ensure that only one rcu_barrier() is using the
global completion and counters at a time, which are initialized on lines
6 and 7. Line 8 causes each CPU to invoke rcu_barrier_func(), which is
shown below. Note that the final "1" in on_each_cpu()'s argument list
ensures that all the calls to rcu_barrier_func() will have completed
before on_each_cpu() returns. Line 9 then waits for the completion.
This code was rewritten in 2008 to support rcu_barrier_bh() and
rcu_barrier_sched() in addition to the original rcu_barrier().
The rcu_barrier_func() runs on each CPU, where it invokes call_rcu()
to post an RCU callback, as follows:
1 static void rcu_barrier_func(void *notused)
2 {
3 int cpu = smp_processor_id();
4 struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
5 struct rcu_head *head;
6
7 head = &rdp->barrier;
8 atomic_inc(&rcu_barrier_cpu_count);
9 call_rcu(head, rcu_barrier_callback);
10 }
Lines 3 and 4 locate RCU's internal per-CPU rcu_data structure,
which contains the struct rcu_head that needed for the later call to
call_rcu(). Line 7 picks up a pointer to this struct rcu_head, and line
8 increments a global counter. This counter will later be decremented
by the callback. Line 9 then registers the rcu_barrier_callback() on
the current CPU's queue.
The rcu_barrier_callback() function simply atomically decrements the
rcu_barrier_cpu_count variable and finalizes the completion when it
reaches zero, as follows:
1 static void rcu_barrier_callback(struct rcu_head *notused)
2 {
3 if (atomic_dec_and_test(&rcu_barrier_cpu_count))
4 complete(&rcu_barrier_completion);
5 }
Quick Quiz #3: What happens if CPU 0's rcu_barrier_func() executes
immediately (thus incrementing rcu_barrier_cpu_count to the
value one), but the other CPU's rcu_barrier_func() invocations
are delayed for a full grace period? Couldn't this result in
rcu_barrier() returning prematurely?
rcu_barrier() Summary
The rcu_barrier() primitive has seen relatively little use, since most
code using RCU is in the core kernel rather than in modules. However, if
you are using RCU from an unloadable module, you need to use rcu_barrier()
so that your module may be safely unloaded.
Answers to Quick Quizzes
Quick Quiz #1: Why is there no srcu_barrier()?
Answer: Since there is no call_srcu(), there can be no outstanding SRCU
callbacks. Therefore, there is no need to wait for them.
Quick Quiz #2: Is there any other situation where rcu_barrier() might
be required?
Answer: Interestingly enough, rcu_barrier() was not originally
implemented for module unloading. Nikita Danilov was using
RCU in a filesystem, which resulted in a similar situation at
filesystem-unmount time. Dipankar Sarma coded up rcu_barrier()
in response, so that Nikita could invoke it during the
filesystem-unmount process.
Much later, yours truly hit the RCU module-unload problem when
implementing rcutorture, and found that rcu_barrier() solves
this problem as well.
Quick Quiz #3: What happens if CPU 0's rcu_barrier_func() executes
immediately (thus incrementing rcu_barrier_cpu_count to the
value one), but the other CPU's rcu_barrier_func() invocations
are delayed for a full grace period? Couldn't this result in
rcu_barrier() returning prematurely?
Answer: This cannot happen. The reason is that on_each_cpu() has its last
argument, the wait flag, set to "1". This flag is passed through
to smp_call_function() and further to smp_call_function_on_cpu(),
causing this latter to spin until the cross-CPU invocation of
rcu_barrier_func() has completed. This by itself would prevent
a grace period from completing on non-CONFIG_PREEMPT kernels,
since each CPU must undergo a context switch (or other quiescent
state) before the grace period can complete. However, this is
of no use in CONFIG_PREEMPT kernels.
Therefore, on_each_cpu() disables preemption across its call
to smp_call_function() and also across the local call to
rcu_barrier_func(). This prevents the local CPU from context
switching, again preventing grace periods from completing. This
means that all CPUs have executed rcu_barrier_func() before
the first rcu_barrier_callback() can possibly execute, in turn
preventing rcu_barrier_cpu_count from prematurely reaching zero.
Currently, -rt implementations of RCU keep but a single global
queue for RCU callbacks, and thus do not suffer from this
problem. However, when the -rt RCU eventually does have per-CPU
callback queues, things will have to change. One simple change
is to add an rcu_read_lock() before line 8 of rcu_barrier()
and an rcu_read_unlock() after line 8 of this same function. If
you can think of a better change, please let me know!
Using hlist_nulls to protect read-mostly linked lists and
objects using SLAB_DESTROY_BY_RCU allocations.
Please read the basics in Documentation/RCU/listRCU.txt
Using special makers (called 'nulls') is a convenient way
to solve following problem :
A typical RCU linked list managing objects which are
allocated with SLAB_DESTROY_BY_RCU kmem_cache can
use following algos :
1) Lookup algo
--------------
rcu_read_lock()
begin:
obj = lockless_lookup(key);
if (obj) {
if (!try_get_ref(obj)) // might fail for free objects
goto begin;
/*
* Because a writer could delete object, and a writer could
* reuse these object before the RCU grace period, we
* must check key after geting the reference on object
*/
if (obj->key != key) { // not the object we expected
put_ref(obj);
goto begin;
}
}
rcu_read_unlock();
Beware that lockless_lookup(key) cannot use traditional hlist_for_each_entry_rcu()
but a version with an additional memory barrier (smp_rmb())
lockless_lookup(key)
{
struct hlist_node *node, *next;
for (pos = rcu_dereference((head)->first);
pos && ({ next = pos->next; smp_rmb(); prefetch(next); 1; }) &&
({ tpos = hlist_entry(pos, typeof(*tpos), member); 1; });
pos = rcu_dereference(next))
if (obj->key == key)
return obj;
return NULL;
And note the traditional hlist_for_each_entry_rcu() misses this smp_rmb() :
struct hlist_node *node;
for (pos = rcu_dereference((head)->first);
pos && ({ prefetch(pos->next); 1; }) &&
({ tpos = hlist_entry(pos, typeof(*tpos), member); 1; });
pos = rcu_dereference(pos->next))
if (obj->key == key)
return obj;
return NULL;
}
Quoting Corey Minyard :
"If the object is moved from one list to another list in-between the
time the hash is calculated and the next field is accessed, and the
object has moved to the end of a new list, the traversal will not
complete properly on the list it should have, since the object will
be on the end of the new list and there's not a way to tell it's on a
new list and restart the list traversal. I think that this can be
solved by pre-fetching the "next" field (with proper barriers) before
checking the key."
2) Insert algo :
----------------
We need to make sure a reader cannot read the new 'obj->obj_next' value
and previous value of 'obj->key'. Or else, an item could be deleted
from a chain, and inserted into another chain. If new chain was empty
before the move, 'next' pointer is NULL, and lockless reader can
not detect it missed following items in original chain.
/*
* Please note that new inserts are done at the head of list,
* not in the middle or end.
*/
obj = kmem_cache_alloc(...);
lock_chain(); // typically a spin_lock()
obj->key = key;
atomic_inc(&obj->refcnt);
/*
* we need to make sure obj->key is updated before obj->next
*/
smp_wmb();
hlist_add_head_rcu(&obj->obj_node, list);
unlock_chain(); // typically a spin_unlock()
3) Remove algo
--------------
Nothing special here, we can use a standard RCU hlist deletion.
But thanks to SLAB_DESTROY_BY_RCU, beware a deleted object can be reused
very very fast (before the end of RCU grace period)
if (put_last_reference_on(obj) {
lock_chain(); // typically a spin_lock()
hlist_del_init_rcu(&obj->obj_node);
unlock_chain(); // typically a spin_unlock()
kmem_cache_free(cachep, obj);
}
--------------------------------------------------------------------------
With hlist_nulls we can avoid extra smp_rmb() in lockless_lookup()
and extra smp_wmb() in insert function.
For example, if we choose to store the slot number as the 'nulls'
end-of-list marker for each slot of the hash table, we can detect
a race (some writer did a delete and/or a move of an object
to another chain) checking the final 'nulls' value if
the lookup met the end of chain. If final 'nulls' value
is not the slot number, then we must restart the lookup at
the begining. If the object was moved to same chain,
then the reader doesnt care : It might eventually
scan the list again without harm.
1) lookup algo
head = &table[slot];
rcu_read_lock();
begin:
hlist_nulls_for_each_entry_rcu(obj, node, head, member) {
if (obj->key == key) {
if (!try_get_ref(obj)) // might fail for free objects
goto begin;
if (obj->key != key) { // not the object we expected
put_ref(obj);
goto begin;
}
goto out;
}
/*
* if the nulls value we got at the end of this lookup is
* not the expected one, we must restart lookup.
* We probably met an item that was moved to another chain.
*/
if (get_nulls_value(node) != slot)
goto begin;
obj = NULL;
out:
rcu_read_unlock();
2) Insert function :
--------------------
/*
* Please note that new inserts are done at the head of list,
* not in the middle or end.
*/
obj = kmem_cache_alloc(cachep);
lock_chain(); // typically a spin_lock()
obj->key = key;
atomic_set(&obj->refcnt, 1);
/*
* insert obj in RCU way (readers might be traversing chain)
*/
hlist_nulls_add_head_rcu(&obj->obj_node, list);
unlock_chain(); // typically a spin_unlock()
CONFIG_RCU_TRACE debugfs Files and Formats
The rcupreempt and rcutree implementations of RCU provide debugfs trace
output that summarizes counters and state. This information is useful for
debugging RCU itself, and can sometimes also help to debug abuses of RCU.
Note that the rcuclassic implementation of RCU does not provide debugfs
trace output.
The following sections describe the debugfs files and formats for
preemptable RCU (rcupreempt) and hierarchical RCU (rcutree).
Preemptable RCU debugfs Files and Formats
This implementation of RCU provides three debugfs files under the
top-level directory RCU: rcu/rcuctrs (which displays the per-CPU
counters used by preemptable RCU) rcu/rcugp (which displays grace-period
counters), and rcu/rcustats (which internal counters for debugging RCU).
The output of "cat rcu/rcuctrs" looks as follows:
CPU last cur F M
0 5 -5 0 0
1 -1 0 0 0
2 0 1 0 0
3 0 1 0 0
4 0 1 0 0
5 0 1 0 0
6 0 2 0 0
7 0 -1 0 0
8 0 1 0 0
ggp = 26226, state = waitzero
The per-CPU fields are as follows:
o "CPU" gives the CPU number. Offline CPUs are not displayed.
o "last" gives the value of the counter that is being decremented
for the current grace period phase. In the example above,
the counters sum to 4, indicating that there are still four
RCU read-side critical sections still running that started
before the last counter flip.
o "cur" gives the value of the counter that is currently being
both incremented (by rcu_read_lock()) and decremented (by
rcu_read_unlock()). In the example above, the counters sum to
1, indicating that there is only one RCU read-side critical section
still running that started after the last counter flip.
o "F" indicates whether RCU is waiting for this CPU to acknowledge
a counter flip. In the above example, RCU is not waiting on any,
which is consistent with the state being "waitzero" rather than
"waitack".
o "M" indicates whether RCU is waiting for this CPU to execute a
memory barrier. In the above example, RCU is not waiting on any,
which is consistent with the state being "waitzero" rather than
"waitmb".
o "ggp" is the global grace-period counter.
o "state" is the RCU state, which can be one of the following:
o "idle": there is no grace period in progress.
o "waitack": RCU just incremented the global grace-period
counter, which has the effect of reversing the roles of
the "last" and "cur" counters above, and is waiting for
all the CPUs to acknowledge the flip. Once the flip has
been acknowledged, CPUs will no longer be incrementing
what are now the "last" counters, so that their sum will
decrease monotonically down to zero.
o "waitzero": RCU is waiting for the sum of the "last" counters
to decrease to zero.
o "waitmb": RCU is waiting for each CPU to execute a memory
barrier, which ensures that instructions from a given CPU's
last RCU read-side critical section cannot be reordered
with instructions following the memory-barrier instruction.
The output of "cat rcu/rcugp" looks as follows:
oldggp=48870 newggp=48873
Note that reading from this file provokes a synchronize_rcu(). The
"oldggp" value is that of "ggp" from rcu/rcuctrs above, taken before
executing the synchronize_rcu(), and the "newggp" value is also the
"ggp" value, but taken after the synchronize_rcu() command returns.
The output of "cat rcu/rcugp" looks as follows:
na=1337955 nl=40 wa=1337915 wl=44 da=1337871 dl=0 dr=1337871 di=1337871
1=50989 e1=6138 i1=49722 ie1=82 g1=49640 a1=315203 ae1=265563 a2=49640
z1=1401244 ze1=1351605 z2=49639 m1=5661253 me1=5611614 m2=49639
These are counters tracking internal preemptable-RCU events, however,
some of them may be useful for debugging algorithms using RCU. In
particular, the "nl", "wl", and "dl" values track the number of RCU
callbacks in various states. The fields are as follows:
o "na" is the total number of RCU callbacks that have been enqueued
since boot.
o "nl" is the number of RCU callbacks waiting for the previous
grace period to end so that they can start waiting on the next
grace period.
o "wa" is the total number of RCU callbacks that have started waiting
for a grace period since boot. "na" should be roughly equal to
"nl" plus "wa".
o "wl" is the number of RCU callbacks currently waiting for their
grace period to end.
o "da" is the total number of RCU callbacks whose grace periods
have completed since boot. "wa" should be roughly equal to
"wl" plus "da".
o "dr" is the total number of RCU callbacks that have been removed
from the list of callbacks ready to invoke. "dr" should be roughly
equal to "da".
o "di" is the total number of RCU callbacks that have been invoked
since boot. "di" should be roughly equal to "da", though some
early versions of preemptable RCU had a bug so that only the
last CPU's count of invocations was displayed, rather than the
sum of all CPU's counts.
o "1" is the number of calls to rcu_try_flip(). This should be
roughly equal to the sum of "e1", "i1", "a1", "z1", and "m1"
described below. In other words, the number of times that
the state machine is visited should be equal to the sum of the
number of times that each state is visited plus the number of
times that the state-machine lock acquisition failed.
o "e1" is the number of times that rcu_try_flip() was unable to
acquire the fliplock.
o "i1" is the number of calls to rcu_try_flip_idle().
o "ie1" is the number of times rcu_try_flip_idle() exited early
due to the calling CPU having no work for RCU.
o "g1" is the number of times that rcu_try_flip_idle() decided
to start a new grace period. "i1" should be roughly equal to
"ie1" plus "g1".
o "a1" is the number of calls to rcu_try_flip_waitack().
o "ae1" is the number of times that rcu_try_flip_waitack() found
that at least one CPU had not yet acknowledge the new grace period
(AKA "counter flip").
o "a2" is the number of time rcu_try_flip_waitack() found that
all CPUs had acknowledged. "a1" should be roughly equal to
"ae1" plus "a2". (This particular output was collected on
a 128-CPU machine, hence the smaller-than-usual fraction of
calls to rcu_try_flip_waitack() finding all CPUs having already
acknowledged.)
o "z1" is the number of calls to rcu_try_flip_waitzero().
o "ze1" is the number of times that rcu_try_flip_waitzero() found
that not all of the old RCU read-side critical sections had
completed.
o "z2" is the number of times that rcu_try_flip_waitzero() finds
the sum of the counters equal to zero, in other words, that
all of the old RCU read-side critical sections had completed.
The value of "z1" should be roughly equal to "ze1" plus
"z2".
o "m1" is the number of calls to rcu_try_flip_waitmb().
o "me1" is the number of times that rcu_try_flip_waitmb() finds
that at least one CPU has not yet executed a memory barrier.
o "m2" is the number of times that rcu_try_flip_waitmb() finds that
all CPUs have executed a memory barrier.
Hierarchical RCU debugfs Files and Formats
This implementation of RCU provides three debugfs files under the
top-level directory RCU: rcu/rcudata (which displays fields in struct
rcu_data), rcu/rcugp (which displays grace-period counters), and
rcu/rcuhier (which displays the struct rcu_node hierarchy).
The output of "cat rcu/rcudata" looks as follows:
rcu:
0 c=4011 g=4012 pq=1 pqc=4011 qp=0 rpfq=1 rp=3c2a dt=23301/73 dn=2 df=1882 of=0 ri=2126 ql=2 b=10
1 c=4011 g=4012 pq=1 pqc=4011 qp=0 rpfq=3 rp=39a6 dt=78073/1 dn=2 df=1402 of=0 ri=1875 ql=46 b=10
2 c=4010 g=4010 pq=1 pqc=4010 qp=0 rpfq=-5 rp=1d12 dt=16646/0 dn=2 df=3140 of=0 ri=2080 ql=0 b=10
3 c=4012 g=4013 pq=1 pqc=4012 qp=1 rpfq=3 rp=2b50 dt=21159/1 dn=2 df=2230 of=0 ri=1923 ql=72 b=10
4 c=4012 g=4013 pq=1 pqc=4012 qp=1 rpfq=3 rp=1644 dt=5783/1 dn=2 df=3348 of=0 ri=2805 ql=7 b=10
5 c=4012 g=4013 pq=0 pqc=4011 qp=1 rpfq=3 rp=1aac dt=5879/1 dn=2 df=3140 of=0 ri=2066 ql=10 b=10
6 c=4012 g=4013 pq=1 pqc=4012 qp=1 rpfq=3 rp=ed8 dt=5847/1 dn=2 df=3797 of=0 ri=1266 ql=10 b=10
7 c=4012 g=4013 pq=1 pqc=4012 qp=1 rpfq=3 rp=1fa2 dt=6199/1 dn=2 df=2795 of=0 ri=2162 ql=28 b=10
rcu_bh:
0 c=-268 g=-268 pq=1 pqc=-268 qp=0 rpfq=-145 rp=21d6 dt=23301/73 dn=2 df=0 of=0 ri=0 ql=0 b=10
1 c=-268 g=-268 pq=1 pqc=-268 qp=1 rpfq=-170 rp=20ce dt=78073/1 dn=2 df=26 of=0 ri=5 ql=0 b=10
2 c=-268 g=-268 pq=1 pqc=-268 qp=1 rpfq=-83 rp=fbd dt=16646/0 dn=2 df=28 of=0 ri=4 ql=0 b=10
3 c=-268 g=-268 pq=1 pqc=-268 qp=0 rpfq=-105 rp=178c dt=21159/1 dn=2 df=28 of=0 ri=2 ql=0 b=10
4 c=-268 g=-268 pq=1 pqc=-268 qp=1 rpfq=-30 rp=b54 dt=5783/1 dn=2 df=32 of=0 ri=0 ql=0 b=10
5 c=-268 g=-268 pq=1 pqc=-268 qp=1 rpfq=-29 rp=df5 dt=5879/1 dn=2 df=30 of=0 ri=3 ql=0 b=10
6 c=-268 g=-268 pq=1 pqc=-268 qp=1 rpfq=-28 rp=788 dt=5847/1 dn=2 df=32 of=0 ri=0 ql=0 b=10
7 c=-268 g=-268 pq=1 pqc=-268 qp=1 rpfq=-53 rp=1098 dt=6199/1 dn=2 df=30 of=0 ri=3 ql=0 b=10
The first section lists the rcu_data structures for rcu, the second for
rcu_bh. Each section has one line per CPU, or eight for this 8-CPU system.
The fields are as follows:
o The number at the beginning of each line is the CPU number.
CPUs numbers followed by an exclamation mark are offline,
but have been online at least once since boot. There will be
no output for CPUs that have never been online, which can be
a good thing in the surprisingly common case where NR_CPUS is
substantially larger than the number of actual CPUs.
o "c" is the count of grace periods that this CPU believes have
completed. CPUs in dynticks idle mode may lag quite a ways
behind, for example, CPU 4 under "rcu" above, which has slept
through the past 25 RCU grace periods. It is not unusual to
see CPUs lagging by thousands of grace periods.
o "g" is the count of grace periods that this CPU believes have
started. Again, CPUs in dynticks idle mode may lag behind.
If the "c" and "g" values are equal, this CPU has already
reported a quiescent state for the last RCU grace period that
it is aware of, otherwise, the CPU believes that it owes RCU a
quiescent state.
o "pq" indicates that this CPU has passed through a quiescent state
for the current grace period. It is possible for "pq" to be
"1" and "c" different than "g", which indicates that although
the CPU has passed through a quiescent state, either (1) this
CPU has not yet reported that fact, (2) some other CPU has not
yet reported for this grace period, or (3) both.
o "pqc" indicates which grace period the last-observed quiescent
state for this CPU corresponds to. This is important for handling
the race between CPU 0 reporting an extended dynticks-idle
quiescent state for CPU 1 and CPU 1 suddenly waking up and
reporting its own quiescent state. If CPU 1 was the last CPU
for the current grace period, then the CPU that loses this race
will attempt to incorrectly mark CPU 1 as having checked in for
the next grace period!
o "qp" indicates that RCU still expects a quiescent state from
this CPU.
o "rpfq" is the number of rcu_pending() calls on this CPU required
to induce this CPU to invoke force_quiescent_state().
o "rp" is low-order four hex digits of the count of how many times
rcu_pending() has been invoked on this CPU.
o "dt" is the current value of the dyntick counter that is incremented
when entering or leaving dynticks idle state, either by the
scheduler or by irq. The number after the "/" is the interrupt
nesting depth when in dyntick-idle state, or one greater than
the interrupt-nesting depth otherwise.
This field is displayed only for CONFIG_NO_HZ kernels.
o "dn" is the current value of the dyntick counter that is incremented
when entering or leaving dynticks idle state via NMI. If both
the "dt" and "dn" values are even, then this CPU is in dynticks
idle mode and may be ignored by RCU. If either of these two
counters is odd, then RCU must be alert to the possibility of
an RCU read-side critical section running on this CPU.
This field is displayed only for CONFIG_NO_HZ kernels.
o "df" is the number of times that some other CPU has forced a
quiescent state on behalf of this CPU due to this CPU being in
dynticks-idle state.
This field is displayed only for CONFIG_NO_HZ kernels.
o "of" is the number of times that some other CPU has forced a
quiescent state on behalf of this CPU due to this CPU being
offline. In a perfect world, this might neve happen, but it
turns out that offlining and onlining a CPU can take several grace
periods, and so there is likely to be an extended period of time
when RCU believes that the CPU is online when it really is not.
Please note that erring in the other direction (RCU believing a
CPU is offline when it is really alive and kicking) is a fatal
error, so it makes sense to err conservatively.
o "ri" is the number of times that RCU has seen fit to send a
reschedule IPI to this CPU in order to get it to report a
quiescent state.
o "ql" is the number of RCU callbacks currently residing on
this CPU. This is the total number of callbacks, regardless
of what state they are in (new, waiting for grace period to
start, waiting for grace period to end, ready to invoke).
o "b" is the batch limit for this CPU. If more than this number
of RCU callbacks is ready to invoke, then the remainder will
be deferred.
The output of "cat rcu/rcugp" looks as follows:
rcu: completed=33062 gpnum=33063
rcu_bh: completed=464 gpnum=464
Again, this output is for both "rcu" and "rcu_bh". The fields are
taken from the rcu_state structure, and are as follows:
o "completed" is the number of grace periods that have completed.
It is comparable to the "c" field from rcu/rcudata in that a
CPU whose "c" field matches the value of "completed" is aware
that the corresponding RCU grace period has completed.
o "gpnum" is the number of grace periods that have started. It is
comparable to the "g" field from rcu/rcudata in that a CPU
whose "g" field matches the value of "gpnum" is aware that the
corresponding RCU grace period has started.
If these two fields are equal (as they are for "rcu_bh" above),
then there is no grace period in progress, in other words, RCU
is idle. On the other hand, if the two fields differ (as they
do for "rcu" above), then an RCU grace period is in progress.
The output of "cat rcu/rcuhier" looks as follows, with very long lines:
c=6902 g=6903 s=2 jfq=3 j=72c7 nfqs=13142/nfqsng=0(13142) fqlh=6
1/1 0:127 ^0
3/3 0:35 ^0 0/0 36:71 ^1 0/0 72:107 ^2 0/0 108:127 ^3
3/3f 0:5 ^0 2/3 6:11 ^1 0/0 12:17 ^2 0/0 18:23 ^3 0/0 24:29 ^4 0/0 30:35 ^5 0/0 36:41 ^0 0/0 42:47 ^1 0/0 48:53 ^2 0/0 54:59 ^3 0/0 60:65 ^4 0/0 66:71 ^5 0/0 72:77 ^0 0/0 78:83 ^1 0/0 84:89 ^2 0/0 90:95 ^3 0/0 96:101 ^4 0/0 102:107 ^5 0/0 108:113 ^0 0/0 114:119 ^1 0/0 120:125 ^2 0/0 126:127 ^3
rcu_bh:
c=-226 g=-226 s=1 jfq=-5701 j=72c7 nfqs=88/nfqsng=0(88) fqlh=0
0/1 0:127 ^0
0/3 0:35 ^0 0/0 36:71 ^1 0/0 72:107 ^2 0/0 108:127 ^3
0/3f 0:5 ^0 0/3 6:11 ^1 0/0 12:17 ^2 0/0 18:23 ^3 0/0 24:29 ^4 0/0 30:35 ^5 0/0 36:41 ^0 0/0 42:47 ^1 0/0 48:53 ^2 0/0 54:59 ^3 0/0 60:65 ^4 0/0 66:71 ^5 0/0 72:77 ^0 0/0 78:83 ^1 0/0 84:89 ^2 0/0 90:95 ^3 0/0 96:101 ^4 0/0 102:107 ^5 0/0 108:113 ^0 0/0 114:119 ^1 0/0 120:125 ^2 0/0 126:127 ^3
This is once again split into "rcu" and "rcu_bh" portions. The fields are
as follows:
o "c" is exactly the same as "completed" under rcu/rcugp.
o "g" is exactly the same as "gpnum" under rcu/rcugp.
o "s" is the "signaled" state that drives force_quiescent_state()'s
state machine.
o "jfq" is the number of jiffies remaining for this grace period
before force_quiescent_state() is invoked to help push things
along. Note that CPUs in dyntick-idle mode thoughout the grace
period will not report on their own, but rather must be check by
some other CPU via force_quiescent_state().
o "j" is the low-order four hex digits of the jiffies counter.
Yes, Paul did run into a number of problems that turned out to
be due to the jiffies counter no longer counting. Why do you ask?
o "nfqs" is the number of calls to force_quiescent_state() since
boot.
o "nfqsng" is the number of useless calls to force_quiescent_state(),
where there wasn't actually a grace period active. This can
happen due to races. The number in parentheses is the difference
between "nfqs" and "nfqsng", or the number of times that
force_quiescent_state() actually did some real work.
o "fqlh" is the number of calls to force_quiescent_state() that
exited immediately (without even being counted in nfqs above)
due to contention on ->fqslock.
o Each element of the form "1/1 0:127 ^0" represents one struct
rcu_node. Each line represents one level of the hierarchy, from
root to leaves. It is best to think of the rcu_data structures
as forming yet another level after the leaves. Note that there
might be either one, two, or three levels of rcu_node structures,
depending on the relationship between CONFIG_RCU_FANOUT and
CONFIG_NR_CPUS.
o The numbers separated by the "/" are the qsmask followed
by the qsmaskinit. The qsmask will have one bit
set for each entity in the next lower level that
has not yet checked in for the current grace period.
The qsmaskinit will have one bit for each entity that is
currently expected to check in during each grace period.
The value of qsmaskinit is assigned to that of qsmask
at the beginning of each grace period.
For example, for "rcu", the qsmask of the first entry
of the lowest level is 0x14, meaning that we are still
waiting for CPUs 2 and 4 to check in for the current
grace period.
o The numbers separated by the ":" are the range of CPUs
served by this struct rcu_node. This can be helpful
in working out how the hierarchy is wired together.
For example, the first entry at the lowest level shows
"0:5", indicating that it covers CPUs 0 through 5.
o The number after the "^" indicates the bit in the
next higher level rcu_node structure that this
rcu_node structure corresponds to.
For example, the first entry at the lowest level shows
"^0", indicating that it corresponds to bit zero in
the first entry at the middle level.
......@@ -392,6 +392,10 @@ int main(int argc, char *argv[])
goto err;
}
}
if (!maskset && !tid && !containerset) {
usage();
goto err;
}
do {
int i;
......
MFP Configuration for PXA2xx/PXA3xx Processors
Eric Miao <eric.miao@marvell.com>
MFP stands for Multi-Function Pin, which is the pin-mux logic on PXA3xx and
later PXA series processors. This document describes the existing MFP API,
and how board/platform driver authors could make use of it.
Basic Concept
===============
Unlike the GPIO alternate function settings on PXA25x and PXA27x, a new MFP
mechanism is introduced from PXA3xx to completely move the pin-mux functions
out of the GPIO controller. In addition to pin-mux configurations, the MFP
also controls the low power state, driving strength, pull-up/down and event
detection of each pin. Below is a diagram of internal connections between
the MFP logic and the remaining SoC peripherals:
+--------+
| |--(GPIO19)--+
| GPIO | |
| |--(GPIO...) |
+--------+ |
| +---------+
+--------+ +------>| |
| PWM2 |--(PWM_OUT)-------->| MFP |
+--------+ +------>| |-------> to external PAD
| +---->| |
+--------+ | | +-->| |
| SSP2 |---(TXD)----+ | | +---------+
+--------+ | |
| |
+--------+ | |
| Keypad |--(MKOUT4)----+ |
+--------+ |
|
+--------+ |
| UART2 |---(TXD)--------+
+--------+
NOTE: the external pad is named as MFP_PIN_GPIO19, it doesn't necessarily
mean it's dedicated for GPIO19, only as a hint that internally this pin
can be routed from GPIO19 of the GPIO controller.
To better understand the change from PXA25x/PXA27x GPIO alternate function
to this new MFP mechanism, here are several key points:
1. GPIO controller on PXA3xx is now a dedicated controller, same as other
internal controllers like PWM, SSP and UART, with 128 internal signals
which can be routed to external through one or more MFPs (e.g. GPIO<0>
can be routed through either MFP_PIN_GPIO0 as well as MFP_PIN_GPIO0_2,
see arch/arm/mach-pxa/mach/include/mfp-pxa300.h)
2. Alternate function configuration is removed from this GPIO controller,
the remaining functions are pure GPIO-specific, i.e.
- GPIO signal level control
- GPIO direction control
- GPIO level change detection
3. Low power state for each pin is now controlled by MFP, this means the
PGSRx registers on PXA2xx are now useless on PXA3xx
4. Wakeup detection is now controlled by MFP, PWER does not control the
wakeup from GPIO(s) any more, depending on the sleeping state, ADxER
(as defined in pxa3xx-regs.h) controls the wakeup from MFP
NOTE: with such a clear separation of MFP and GPIO, by GPIO<xx> we normally
mean it is a GPIO signal, and by MFP<xxx> or pin xxx, we mean a physical
pad (or ball).
MFP API Usage
===============
For board code writers, here are some guidelines:
1. include ONE of the following header files in your <board>.c:
- #include <mach/mfp-pxa25x.h>
- #include <mach/mfp-pxa27x.h>
- #include <mach/mfp-pxa300.h>
- #include <mach/mfp-pxa320.h>
- #include <mach/mfp-pxa930.h>
NOTE: only one file in your <board>.c, depending on the processors used,
because pin configuration definitions may conflict in these file (i.e.
same name, different meaning and settings on different processors). E.g.
for zylonite platform, which support both PXA300/PXA310 and PXA320, two
separate files are introduced: zylonite_pxa300.c and zylonite_pxa320.c
(in addition to handle MFP configuration differences, they also handle
the other differences between the two combinations).
NOTE: PXA300 and PXA310 are almost identical in pin configurations (with
PXA310 supporting some additional ones), thus the difference is actually
covered in a single mfp-pxa300.h.
2. prepare an array for the initial pin configurations, e.g.:
static unsigned long mainstone_pin_config[] __initdata = {
/* Chip Select */
GPIO15_nCS_1,
/* LCD - 16bpp Active TFT */
GPIOxx_TFT_LCD_16BPP,
GPIO16_PWM0_OUT, /* Backlight */
/* MMC */
GPIO32_MMC_CLK,
GPIO112_MMC_CMD,
GPIO92_MMC_DAT_0,
GPIO109_MMC_DAT_1,
GPIO110_MMC_DAT_2,
GPIO111_MMC_DAT_3,
...
/* GPIO */
GPIO1_GPIO | WAKEUP_ON_EDGE_BOTH,
};
a) once the pin configurations are passed to pxa{2xx,3xx}_mfp_config(),
and written to the actual registers, they are useless and may discard,
adding '__initdata' will help save some additional bytes here.
b) when there is only one possible pin configurations for a component,
some simplified definitions can be used, e.g. GPIOxx_TFT_LCD_16BPP on
PXA25x and PXA27x processors
c) if by board design, a pin can be configured to wake up the system
from low power state, it can be 'OR'ed with any of:
WAKEUP_ON_EDGE_BOTH
WAKEUP_ON_EDGE_RISE
WAKEUP_ON_EDGE_FALL
WAKEUP_ON_LEVEL_HIGH - specifically for enabling of keypad GPIOs,
to indicate that this pin has the capability of wake-up the system,
and on which edge(s). This, however, doesn't necessarily mean the
pin _will_ wakeup the system, it will only when set_irq_wake() is
invoked with the corresponding GPIO IRQ (GPIO_IRQ(xx) or gpio_to_irq())
and eventually calls gpio_set_wake() for the actual register setting.
d) although PXA3xx MFP supports edge detection on each pin, the
internal logic will only wakeup the system when those specific bits
in ADxER registers are set, which can be well mapped to the
corresponding peripheral, thus set_irq_wake() can be called with
the peripheral IRQ to enable the wakeup.
MFP on PXA3xx
===============
Every external I/O pad on PXA3xx (excluding those for special purpose) has
one MFP logic associated, and is controlled by one MFP register (MFPR).
The MFPR has the following bit definitions (for PXA300/PXA310/PXA320):
31 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
+-------------------------+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| RESERVED |PS|PU|PD| DRIVE |SS|SD|SO|EC|EF|ER|--| AF_SEL |
+-------------------------+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
Bit 3: RESERVED
Bit 4: EDGE_RISE_EN - enable detection of rising edge on this pin
Bit 5: EDGE_FALL_EN - enable detection of falling edge on this pin
Bit 6: EDGE_CLEAR - disable edge detection on this pin
Bit 7: SLEEP_OE_N - enable outputs during low power modes
Bit 8: SLEEP_DATA - output data on the pin during low power modes
Bit 9: SLEEP_SEL - selection control for low power modes signals
Bit 13: PULLDOWN_EN - enable the internal pull-down resistor on this pin
Bit 14: PULLUP_EN - enable the internal pull-up resistor on this pin
Bit 15: PULL_SEL - pull state controlled by selected alternate function
(0) or by PULL{UP,DOWN}_EN bits (1)
Bit 0 - 2: AF_SEL - alternate function selection, 8 possibilities, from 0-7
Bit 10-12: DRIVE - drive strength and slew rate
0b000 - fast 1mA
0b001 - fast 2mA
0b002 - fast 3mA
0b003 - fast 4mA
0b004 - slow 6mA
0b005 - fast 6mA
0b006 - slow 10mA
0b007 - fast 10mA
MFP Design for PXA2xx/PXA3xx
==============================
Due to the difference of pin-mux handling between PXA2xx and PXA3xx, a unified
MFP API is introduced to cover both series of processors.
The basic idea of this design is to introduce definitions for all possible pin
configurations, these definitions are processor and platform independent, and
the actual API invoked to convert these definitions into register settings and
make them effective there-after.
Files Involved
--------------
- arch/arm/mach-pxa/include/mach/mfp.h
for
1. Unified pin definitions - enum constants for all configurable pins
2. processor-neutral bit definitions for a possible MFP configuration
- arch/arm/mach-pxa/include/mach/mfp-pxa3xx.h
for PXA3xx specific MFPR register bit definitions and PXA3xx common pin
configurations
- arch/arm/mach-pxa/include/mach/mfp-pxa2xx.h
for PXA2xx specific definitions and PXA25x/PXA27x common pin configurations
- arch/arm/mach-pxa/include/mach/mfp-pxa25x.h
arch/arm/mach-pxa/include/mach/mfp-pxa27x.h
arch/arm/mach-pxa/include/mach/mfp-pxa300.h
arch/arm/mach-pxa/include/mach/mfp-pxa320.h
arch/arm/mach-pxa/include/mach/mfp-pxa930.h
for processor specific definitions
- arch/arm/mach-pxa/mfp-pxa3xx.c
- arch/arm/mach-pxa/mfp-pxa2xx.c
for implementation of the pin configuration to take effect for the actual
processor.
Pin Configuration
-----------------
The following comments are copied from mfp.h (see the actual source code
for most updated info)
/*
* a possible MFP configuration is represented by a 32-bit integer
*
* bit 0.. 9 - MFP Pin Number (1024 Pins Maximum)
* bit 10..12 - Alternate Function Selection
* bit 13..15 - Drive Strength
* bit 16..18 - Low Power Mode State
* bit 19..20 - Low Power Mode Edge Detection
* bit 21..22 - Run Mode Pull State
*
* to facilitate the definition, the following macros are provided
*
* MFP_CFG_DEFAULT - default MFP configuration value, with
* alternate function = 0,
* drive strength = fast 3mA (MFP_DS03X)
* low power mode = default
* edge detection = none
*
* MFP_CFG - default MFPR value with alternate function
* MFP_CFG_DRV - default MFPR value with alternate function and
* pin drive strength
* MFP_CFG_LPM - default MFPR value with alternate function and
* low power mode
* MFP_CFG_X - default MFPR value with alternate function,
* pin drive strength and low power mode
*/
Examples of pin configurations are:
#define GPIO94_SSP3_RXD MFP_CFG_X(GPIO94, AF1, DS08X, FLOAT)
which reads GPIO94 can be configured as SSP3_RXD, with alternate function
selection of 1, driving strength of 0b101, and a float state in low power
modes.
NOTE: this is the default setting of this pin being configured as SSP3_RXD
which can be modified a bit in board code, though it is not recommended to
do so, simply because this default setting is usually carefully encoded,
and is supposed to work in most cases.
Register Settings
-----------------
Register settings on PXA3xx for a pin configuration is actually very
straight-forward, most bits can be converted directly into MFPR value
in a easier way. Two sets of MFPR values are calculated: the run-time
ones and the low power mode ones, to allow different settings.
The conversion from a generic pin configuration to the actual register
settings on PXA2xx is a bit complicated: many registers are involved,
including GAFRx, GPDRx, PGSRx, PWER, PKWR, PFER and PRER. Please see
mfp-pxa2xx.c for how the conversion is made.
March 2008
Jan-Simon Moeller, dl9pf@gmx.de
How to deal with bad memory e.g. reported by memtest86+ ?
#########################################################
There are three possibilities I know of:
1) Reinsert/swap the memory modules
2) Buy new modules (best!) or try to exchange the memory
if you have spare-parts
3) Use BadRAM or memmap
This Howto is about number 3) .
BadRAM
######
BadRAM is the actively developed and available as kernel-patch
here: http://rick.vanrein.org/linux/badram/
For more details see the BadRAM documentation.
memmap
######
memmap is already in the kernel and usable as kernel-parameter at
boot-time. Its syntax is slightly strange and you may need to
calculate the values by yourself!
Syntax to exclude a memory area (see kernel-parameters.txt for details):
memmap=<size>$<address>
Example: memtest86+ reported here errors at address 0x18691458, 0x18698424 and
some others. All had 0x1869xxxx in common, so I chose a pattern of
0x18690000,0xffff0000.
With the numbers of the example above:
memmap=64K$0x18690000
or
memmap=0x10000$0x18690000
......@@ -9,3 +9,6 @@ cachefeatures.txt
Filesystems
- Requirements for mounting the root file system.
bfin-gpio-note.txt
- Notes in developing/using bfin-gpio driver.
/*
* File: Documentation/blackfin/bfin-gpio-note.txt
* Based on:
* Author:
*
* Created: $Id: bfin-gpio-note.txt 2008-11-24 16:42 grafyang $
* Description: This file contains the notes in developing/using bfin-gpio.
*
*
* Rev:
*
* Modified:
* Copyright 2004-2008 Analog Devices Inc.
*
* Bugs: Enter bugs at http://blackfin.uclinux.org/
*
*/
1. Blackfin GPIO introduction
There are many GPIO pins on Blackfin. Most of these pins are muxed to
multi-functions. They can be configured as peripheral, or just as GPIO,
configured to input with interrupt enabled, or output.
For detailed information, please see "arch/blackfin/kernel/bfin_gpio.c",
or the relevant HRM.
2. Avoiding resource conflict
Followed function groups are used to avoiding resource conflict,
- Use the pin as peripheral,
int peripheral_request(unsigned short per, const char *label);
int peripheral_request_list(const unsigned short per[], const char *label);
void peripheral_free(unsigned short per);
void peripheral_free_list(const unsigned short per[]);
- Use the pin as GPIO,
int bfin_gpio_request(unsigned gpio, const char *label);
void bfin_gpio_free(unsigned gpio);
- Use the pin as GPIO interrupt,
int bfin_gpio_irq_request(unsigned gpio, const char *label);
void bfin_gpio_irq_free(unsigned gpio);
The request functions will record the function state for a certain pin,
the free functions will clear it's function state.
Once a pin is requested, it can't be requested again before it is freed by
previous caller, otherwise kernel will dump stacks, and the request
function fail.
These functions are wrapped by other functions, most of the users need not
care.
3. But there are some exceptions
- Kernel permit the identical GPIO be requested both as GPIO and GPIO
interrut.
Some drivers, like gpio-keys, need this behavior. Kernel only print out
warning messages like,
bfin-gpio: GPIO 24 is already reserved by gpio-keys: BTN0, and you are
configuring it as IRQ!
Note: Consider the case that, if there are two drivers need the
identical GPIO, one of them use it as GPIO, the other use it as
GPIO interrupt. This will really cause resource conflict. So if
there is any abnormal driver behavior, please check the bfin-gpio
warning messages.
- Kernel permit the identical GPIO be requested from the same driver twice.
......@@ -914,7 +914,7 @@ I/O scheduler, a.k.a. elevator, is implemented in two layers. Generic dispatch
queue and specific I/O schedulers. Unless stated otherwise, elevator is used
to refer to both parts and I/O scheduler to specific I/O schedulers.
Block layer implements generic dispatch queue in ll_rw_blk.c and elevator.c.
Block layer implements generic dispatch queue in block/*.c.
The generic dispatch queue is responsible for properly ordering barrier
requests, requeueing, handling non-fs requests and all other subtleties.
......@@ -926,8 +926,8 @@ be built inside the kernel. Each queue can choose different one and can also
change to another one dynamically.
A block layer call to the i/o scheduler follows the convention elv_xxx(). This
calls elevator_xxx_fn in the elevator switch (drivers/block/elevator.c). Oh,
xxx and xxx might not match exactly, but use your imagination. If an elevator
calls elevator_xxx_fn in the elevator switch (block/elevator.c). Oh, xxx
and xxx might not match exactly, but use your imagination. If an elevator
doesn't implement a function, the switch does nothing or some minimal house
keeping work.
......
CGROUPS
-------
Written by Paul Menage <menage@google.com> based on Documentation/cpusets.txt
Written by Paul Menage <menage@google.com> based on
Documentation/cgroups/cpusets.txt
Original copyright statements from cpusets.txt:
Portions Copyright (C) 2004 BULL SA.
......@@ -68,7 +69,7 @@ On their own, the only use for cgroups is for simple job
tracking. The intention is that other subsystems hook into the generic
cgroup support to provide new attributes for cgroups, such as
accounting/limiting the resources which processes in a cgroup can
access. For example, cpusets (see Documentation/cpusets.txt) allows
access. For example, cpusets (see Documentation/cgroups/cpusets.txt) allows
you to associate a set of CPUs and a set of memory nodes with the
tasks in each cgroup.
......@@ -227,7 +228,6 @@ Each cgroup is represented by a directory in the cgroup file system
containing the following files describing that cgroup:
- tasks: list of tasks (by pid) attached to that cgroup
- releasable flag: cgroup currently removeable?
- notify_on_release flag: run the release agent on exit?
- release_agent: the path to use for release notifications (this file
exists in the top cgroup only)
......@@ -360,7 +360,7 @@ Now you want to do something with this cgroup.
In this directory you can find several files:
# ls
notify_on_release releasable tasks
notify_on_release tasks
(plus whatever files added by the attached subsystems)
Now attach your shell to this cgroup:
......@@ -479,7 +479,6 @@ newly-created cgroup if an error occurs after this subsystem's
create() method has been called for the new cgroup).
void pre_destroy(struct cgroup_subsys *ss, struct cgroup *cgrp);
(cgroup_mutex held by caller)
Called before checking the reference count on each subsystem. This may
be useful for subsystems which have some extra references even if
......@@ -498,6 +497,7 @@ remain valid while the caller holds cgroup_mutex.
void attach(struct cgroup_subsys *ss, struct cgroup *cgrp,
struct cgroup *old_cgrp, struct task_struct *task)
(cgroup_mutex held by caller)
Called after the task has been attached to the cgroup, to allow any
post-attachment activity that requires memory allocations or blocking.
......@@ -511,6 +511,7 @@ void exit(struct cgroup_subsys *ss, struct task_struct *task)
Called during task exit.
int populate(struct cgroup_subsys *ss, struct cgroup *cgrp)
(cgroup_mutex held by caller)
Called after creation of a cgroup to allow a subsystem to populate
the cgroup directory with file entries. The subsystem should make
......@@ -520,6 +521,7 @@ method can return an error code, the error code is currently not
always handled well.
void post_clone(struct cgroup_subsys *ss, struct cgroup *cgrp)
(cgroup_mutex held by caller)
Called at the end of cgroup_clone() to do any paramater
initialization which might be required before a task could attach. For
......@@ -527,7 +529,7 @@ example in cpusets, no task may attach before 'cpus' and 'mems' are set
up.
void bind(struct cgroup_subsys *ss, struct cgroup *root)
(cgroup_mutex held by caller)
(cgroup_mutex and ss->hierarchy_mutex held by caller)
Called when a cgroup subsystem is rebound to a different hierarchy
and root cgroup. Currently this will only involve movement between
......
CPU Accounting Controller
-------------------------
The CPU accounting controller is used to group tasks using cgroups and
account the CPU usage of these groups of tasks.
The CPU accounting controller supports multi-hierarchy groups. An accounting
group accumulates the CPU usage of all of its child groups and the tasks
directly present in its group.
Accounting groups can be created by first mounting the cgroup filesystem.
# mkdir /cgroups
# mount -t cgroup -ocpuacct none /cgroups
With the above step, the initial or the parent accounting group
becomes visible at /cgroups. At bootup, this group includes all the
tasks in the system. /cgroups/tasks lists the tasks in this cgroup.
/cgroups/cpuacct.usage gives the CPU time (in nanoseconds) obtained by
this group which is essentially the CPU time obtained by all the tasks
in the system.
New accounting groups can be created under the parent group /cgroups.
# cd /cgroups
# mkdir g1
# echo $$ > g1
The above steps create a new group g1 and move the current shell
process (bash) into it. CPU time consumed by this bash and its children
can be obtained from g1/cpuacct.usage and the same is accumulated in
/cgroups/cpuacct.usage also.
Memory Resource Controller(Memcg) Implementation Memo.
Last Updated: 2008/12/15
Base Kernel Version: based on 2.6.28-rc8-mm.
Because VM is getting complex (one of reasons is memcg...), memcg's behavior
is complex. This is a document for memcg's internal behavior.
Please note that implementation details can be changed.
(*) Topics on API should be in Documentation/cgroups/memory.txt)
0. How to record usage ?
2 objects are used.
page_cgroup ....an object per page.
Allocated at boot or memory hotplug. Freed at memory hot removal.
swap_cgroup ... an entry per swp_entry.
Allocated at swapon(). Freed at swapoff().
The page_cgroup has USED bit and double count against a page_cgroup never
occurs. swap_cgroup is used only when a charged page is swapped-out.
1. Charge
a page/swp_entry may be charged (usage += PAGE_SIZE) at
mem_cgroup_newpage_charge()
Called at new page fault and Copy-On-Write.
mem_cgroup_try_charge_swapin()
Called at do_swap_page() (page fault on swap entry) and swapoff.
Followed by charge-commit-cancel protocol. (With swap accounting)
At commit, a charge recorded in swap_cgroup is removed.
mem_cgroup_cache_charge()
Called at add_to_page_cache()
mem_cgroup_cache_charge_swapin()
Called at shmem's swapin.
mem_cgroup_prepare_migration()
Called before migration. "extra" charge is done and followed by
charge-commit-cancel protocol.
At commit, charge against oldpage or newpage will be committed.
2. Uncharge
a page/swp_entry may be uncharged (usage -= PAGE_SIZE) by
mem_cgroup_uncharge_page()
Called when an anonymous page is fully unmapped. I.e., mapcount goes
to 0. If the page is SwapCache, uncharge is delayed until
mem_cgroup_uncharge_swapcache().
mem_cgroup_uncharge_cache_page()
Called when a page-cache is deleted from radix-tree. If the page is
SwapCache, uncharge is delayed until mem_cgroup_uncharge_swapcache().
mem_cgroup_uncharge_swapcache()
Called when SwapCache is removed from radix-tree. The charge itself
is moved to swap_cgroup. (If mem+swap controller is disabled, no
charge to swap occurs.)
mem_cgroup_uncharge_swap()
Called when swp_entry's refcnt goes down to 0. A charge against swap
disappears.
mem_cgroup_end_migration(old, new)
At success of migration old is uncharged (if necessary), a charge
to new page is committed. At failure, charge to old page is committed.
3. charge-commit-cancel
In some case, we can't know this "charge" is valid or not at charging
(because of races).
To handle such case, there are charge-commit-cancel functions.
mem_cgroup_try_charge_XXX
mem_cgroup_commit_charge_XXX
mem_cgroup_cancel_charge_XXX
these are used in swap-in and migration.
At try_charge(), there are no flags to say "this page is charged".
at this point, usage += PAGE_SIZE.
At commit(), the function checks the page should be charged or not
and set flags or avoid charging.(usage -= PAGE_SIZE)
At cancel(), simply usage -= PAGE_SIZE.
Under below explanation, we assume CONFIG_MEM_RES_CTRL_SWAP=y.
4. Anonymous
Anonymous page is newly allocated at
- page fault into MAP_ANONYMOUS mapping.
- Copy-On-Write.
It is charged right after it's allocated before doing any page table
related operations. Of course, it's uncharged when another page is used
for the fault address.
At freeing anonymous page (by exit() or munmap()), zap_pte() is called
and pages for ptes are freed one by one.(see mm/memory.c). Uncharges
are done at page_remove_rmap() when page_mapcount() goes down to 0.
Another page freeing is by page-reclaim (vmscan.c) and anonymous
pages are swapped out. In this case, the page is marked as
PageSwapCache(). uncharge() routine doesn't uncharge the page marked
as SwapCache(). It's delayed until __delete_from_swap_cache().
4.1 Swap-in.
At swap-in, the page is taken from swap-cache. There are 2 cases.
(a) If the SwapCache is newly allocated and read, it has no charges.
(b) If the SwapCache has been mapped by processes, it has been
charged already.
This swap-in is one of the most complicated work. In do_swap_page(),
following events occur when pte is unchanged.
(1) the page (SwapCache) is looked up.
(2) lock_page()
(3) try_charge_swapin()
(4) reuse_swap_page() (may call delete_swap_cache())
(5) commit_charge_swapin()
(6) swap_free().
Considering following situation for example.
(A) The page has not been charged before (2) and reuse_swap_page()
doesn't call delete_from_swap_cache().
(B) The page has not been charged before (2) and reuse_swap_page()
calls delete_from_swap_cache().
(C) The page has been charged before (2) and reuse_swap_page() doesn't
call delete_from_swap_cache().
(D) The page has been charged before (2) and reuse_swap_page() calls
delete_from_swap_cache().
memory.usage/memsw.usage changes to this page/swp_entry will be
Case (A) (B) (C) (D)
Event
Before (2) 0/ 1 0/ 1 1/ 1 1/ 1
===========================================
(3) +1/+1 +1/+1 +1/+1 +1/+1
(4) - 0/ 0 - -1/ 0
(5) 0/-1 0/ 0 -1/-1 0/ 0
(6) - 0/-1 - 0/-1
===========================================
Result 1/ 1 1/ 1 1/ 1 1/ 1
In any cases, charges to this page should be 1/ 1.
4.2 Swap-out.
At swap-out, typical state transition is below.
(a) add to swap cache. (marked as SwapCache)
swp_entry's refcnt += 1.
(b) fully unmapped.
swp_entry's refcnt += # of ptes.
(c) write back to swap.
(d) delete from swap cache. (remove from SwapCache)
swp_entry's refcnt -= 1.
At (b), the page is marked as SwapCache and not uncharged.
At (d), the page is removed from SwapCache and a charge in page_cgroup
is moved to swap_cgroup.
Finally, at task exit,
(e) zap_pte() is called and swp_entry's refcnt -=1 -> 0.
Here, a charge in swap_cgroup disappears.
5. Page Cache
Page Cache is charged at
- add_to_page_cache_locked().
uncharged at
- __remove_from_page_cache().
The logic is very clear. (About migration, see below)
Note: __remove_from_page_cache() is called by remove_from_page_cache()
and __remove_mapping().
6. Shmem(tmpfs) Page Cache
Memcg's charge/uncharge have special handlers of shmem. The best way
to understand shmem's page state transition is to read mm/shmem.c.
But brief explanation of the behavior of memcg around shmem will be
helpful to understand the logic.
Shmem's page (just leaf page, not direct/indirect block) can be on
- radix-tree of shmem's inode.
- SwapCache.
- Both on radix-tree and SwapCache. This happens at swap-in
and swap-out,
It's charged when...
- A new page is added to shmem's radix-tree.
- A swp page is read. (move a charge from swap_cgroup to page_cgroup)
It's uncharged when
- A page is removed from radix-tree and not SwapCache.
- When SwapCache is removed, a charge is moved to swap_cgroup.
- When swp_entry's refcnt goes down to 0, a charge in swap_cgroup
disappears.
7. Page Migration
One of the most complicated functions is page-migration-handler.
Memcg has 2 routines. Assume that we are migrating a page's contents
from OLDPAGE to NEWPAGE.
Usual migration logic is..
(a) remove the page from LRU.
(b) allocate NEWPAGE (migration target)
(c) lock by lock_page().
(d) unmap all mappings.
(e-1) If necessary, replace entry in radix-tree.
(e-2) move contents of a page.
(f) map all mappings again.
(g) pushback the page to LRU.
(-) OLDPAGE will be freed.
Before (g), memcg should complete all necessary charge/uncharge to
NEWPAGE/OLDPAGE.
The point is....
- If OLDPAGE is anonymous, all charges will be dropped at (d) because
try_to_unmap() drops all mapcount and the page will not be
SwapCache.
- If OLDPAGE is SwapCache, charges will be kept at (g) because
__delete_from_swap_cache() isn't called at (e-1)
- If OLDPAGE is page-cache, charges will be kept at (g) because
remove_from_swap_cache() isn't called at (e-1)
memcg provides following hooks.
- mem_cgroup_prepare_migration(OLDPAGE)
Called after (b) to account a charge (usage += PAGE_SIZE) against
memcg which OLDPAGE belongs to.
- mem_cgroup_end_migration(OLDPAGE, NEWPAGE)
Called after (f) before (g).
If OLDPAGE is used, commit OLDPAGE again. If OLDPAGE is already
charged, a charge by prepare_migration() is automatically canceled.
If NEWPAGE is used, commit NEWPAGE and uncharge OLDPAGE.
But zap_pte() (by exit or munmap) can be called while migration,
we have to check if OLDPAGE/NEWPAGE is a valid page after commit().
8. LRU
Each memcg has its own private LRU. Now, it's handling is under global
VM's control (means that it's handled under global zone->lru_lock).
Almost all routines around memcg's LRU is called by global LRU's
list management functions under zone->lru_lock().
A special function is mem_cgroup_isolate_pages(). This scans
memcg's private LRU and call __isolate_lru_page() to extract a page
from LRU.
(By __isolate_lru_page(), the page is removed from both of global and
private LRU.)
9. Typical Tests.
Tests for racy cases.
9.1 Small limit to memcg.
When you do test to do racy case, it's good test to set memcg's limit
to be very small rather than GB. Many races found in the test under
xKB or xxMB limits.
(Memory behavior under GB and Memory behavior under MB shows very
different situation.)
9.2 Shmem
Historically, memcg's shmem handling was poor and we saw some amount
of troubles here. This is because shmem is page-cache but can be
SwapCache. Test with shmem/tmpfs is always good test.
9.3 Migration
For NUMA, migration is an another special case. To do easy test, cpuset
is useful. Following is a sample script to do migration.
mount -t cgroup -o cpuset none /opt/cpuset
mkdir /opt/cpuset/01
echo 1 > /opt/cpuset/01/cpuset.cpus
echo 0 > /opt/cpuset/01/cpuset.mems
echo 1 > /opt/cpuset/01/cpuset.memory_migrate
mkdir /opt/cpuset/02
echo 1 > /opt/cpuset/02/cpuset.cpus
echo 1 > /opt/cpuset/02/cpuset.mems
echo 1 > /opt/cpuset/02/cpuset.memory_migrate
In above set, when you moves a task from 01 to 02, page migration to
node 0 to node 1 will occur. Following is a script to migrate all
under cpuset.
--
move_task()
{
for pid in $1
do
/bin/echo $pid >$2/tasks 2>/dev/null
echo -n $pid
echo -n " "
done
echo END
}
G1_TASK=`cat ${G1}/tasks`
G2_TASK=`cat ${G2}/tasks`
move_task "${G1_TASK}" ${G2} &
--
9.4 Memory hotplug.
memory hotplug test is one of good test.
to offline memory, do following.
# echo offline > /sys/devices/system/memory/memoryXXX/state
(XXX is the place of memory)
This is an easy way to test page migration, too.
9.5 mkdir/rmdir
When using hierarchy, mkdir/rmdir test should be done.
Use tests like the following.
echo 1 >/opt/cgroup/01/memory/use_hierarchy
mkdir /opt/cgroup/01/child_a
mkdir /opt/cgroup/01/child_b
set limit to 01.
add limit to 01/child_b
run jobs under child_a and child_b
create/delete following groups at random while jobs are running.
/opt/cgroup/01/child_a/child_aa
/opt/cgroup/01/child_b/child_bb
/opt/cgroup/01/child_c
running new jobs in new group is also good.
9.6 Mount with other subsystems.
Mounting with other subsystems is a good test because there is a
race and lock dependency with other cgroup subsystems.
example)
# mount -t cgroup none /cgroup -t cpuset,memory,cpu,devices
and do task move, mkdir, rmdir etc...under this.
Memory Resource Controller
NOTE: The Memory Resource Controller has been generically been referred
to as the memory controller in this document. Do not confuse memory controller
used here with the memory controller that is used in hardware.
Salient features
a. Enable control of both RSS (mapped) and Page Cache (unmapped) pages
b. The infrastructure allows easy addition of other types of memory to control
c. Provides *zero overhead* for non memory controller users
d. Provides a double LRU: global memory pressure causes reclaim from the
global LRU; a cgroup on hitting a limit, reclaims from the per
cgroup LRU
NOTE: Swap Cache (unmapped) is not accounted now.
Benefits and Purpose of the memory controller
The memory controller isolates the memory behaviour of a group of tasks
from the rest of the system. The article on LWN [12] mentions some probable
uses of the memory controller. The memory controller can be used to
a. Isolate an application or a group of applications
Memory hungry applications can be isolated and limited to a smaller
amount of memory.
b. Create a cgroup with limited amount of memory, this can be used
as a good alternative to booting with mem=XXXX.
c. Virtualization solutions can control the amount of memory they want
to assign to a virtual machine instance.
d. A CD/DVD burner could control the amount of memory used by the
rest of the system to ensure that burning does not fail due to lack
of available memory.
e. There are several other use cases, find one or use the controller just
for fun (to learn and hack on the VM subsystem).
1. History
The memory controller has a long history. A request for comments for the memory
controller was posted by Balbir Singh [1]. At the time the RFC was posted
there were several implementations for memory control. The goal of the
RFC was to build consensus and agreement for the minimal features required
for memory control. The first RSS controller was posted by Balbir Singh[2]
in Feb 2007. Pavel Emelianov [3][4][5] has since posted three versions of the
RSS controller. At OLS, at the resource management BoF, everyone suggested
that we handle both page cache and RSS together. Another request was raised
to allow user space handling of OOM. The current memory controller is
at version 6; it combines both mapped (RSS) and unmapped Page
Cache Control [11].
2. Memory Control
Memory is a unique resource in the sense that it is present in a limited
amount. If a task requires a lot of CPU processing, the task can spread
its processing over a period of hours, days, months or years, but with
memory, the same physical memory needs to be reused to accomplish the task.
The memory controller implementation has been divided into phases. These
are:
1. Memory controller
2. mlock(2) controller
3. Kernel user memory accounting and slab control
4. user mappings length controller
The memory controller is the first controller developed.
2.1. Design
The core of the design is a counter called the res_counter. The res_counter
tracks the current memory usage and limit of the group of processes associated
with the controller. Each cgroup has a memory controller specific data
structure (mem_cgroup) associated with it.
2.2. Accounting
+--------------------+
| mem_cgroup |
| (res_counter) |
+--------------------+
/ ^ \
/ | \
+---------------+ | +---------------+
| mm_struct | |.... | mm_struct |
| | | | |
+---------------+ | +---------------+
|
+ --------------+
|
+---------------+ +------+--------+
| page +----------> page_cgroup|
| | | |
+---------------+ +---------------+
(Figure 1: Hierarchy of Accounting)
Figure 1 shows the important aspects of the controller
1. Accounting happens per cgroup
2. Each mm_struct knows about which cgroup it belongs to
3. Each page has a pointer to the page_cgroup, which in turn knows the
cgroup it belongs to
The accounting is done as follows: mem_cgroup_charge() is invoked to setup
the necessary data structures and check if the cgroup that is being charged
is over its limit. If it is then reclaim is invoked on the cgroup.
More details can be found in the reclaim section of this document.
If everything goes well, a page meta-data-structure called page_cgroup is
allocated and associated with the page. This routine also adds the page to
the per cgroup LRU.
2.2.1 Accounting details
All mapped anon pages (RSS) and cache pages (Page Cache) are accounted.
(some pages which never be reclaimable and will not be on global LRU
are not accounted. we just accounts pages under usual vm management.)
RSS pages are accounted at page_fault unless they've already been accounted
for earlier. A file page will be accounted for as Page Cache when it's
inserted into inode (radix-tree). While it's mapped into the page tables of
processes, duplicate accounting is carefully avoided.
A RSS page is unaccounted when it's fully unmapped. A PageCache page is
unaccounted when it's removed from radix-tree.
At page migration, accounting information is kept.
Note: we just account pages-on-lru because our purpose is to control amount
of used pages. not-on-lru pages are tend to be out-of-control from vm view.
2.3 Shared Page Accounting
Shared pages are accounted on the basis of the first touch approach. The
cgroup that first touches a page is accounted for the page. The principle
behind this approach is that a cgroup that aggressively uses a shared
page will eventually get charged for it (once it is uncharged from
the cgroup that brought it in -- this will happen on memory pressure).
Exception: If CONFIG_CGROUP_CGROUP_MEM_RES_CTLR_SWAP is not used..
When you do swapoff and make swapped-out pages of shmem(tmpfs) to
be backed into memory in force, charges for pages are accounted against the
caller of swapoff rather than the users of shmem.
2.4 Swap Extension (CONFIG_CGROUP_MEM_RES_CTLR_SWAP)
Swap Extension allows you to record charge for swap. A swapped-in page is
charged back to original page allocator if possible.
When swap is accounted, following files are added.
- memory.memsw.usage_in_bytes.
- memory.memsw.limit_in_bytes.
usage of mem+swap is limited by memsw.limit_in_bytes.
Note: why 'mem+swap' rather than swap.
The global LRU(kswapd) can swap out arbitrary pages. Swap-out means
to move account from memory to swap...there is no change in usage of
mem+swap.
In other words, when we want to limit the usage of swap without affecting
global LRU, mem+swap limit is better than just limiting swap from OS point
of view.
2.5 Reclaim
Each cgroup maintains a per cgroup LRU that consists of an active
and inactive list. When a cgroup goes over its limit, we first try
to reclaim memory from the cgroup so as to make space for the new
pages that the cgroup has touched. If the reclaim is unsuccessful,
an OOM routine is invoked to select and kill the bulkiest task in the
cgroup.
The reclaim algorithm has not been modified for cgroups, except that
pages that are selected for reclaiming come from the per cgroup LRU
list.
2. Locking
The memory controller uses the following hierarchy
1. zone->lru_lock is used for selecting pages to be isolated
2. mem->per_zone->lru_lock protects the per cgroup LRU (per zone)
3. lock_page_cgroup() is used to protect page->page_cgroup
3. User Interface
0. Configuration
a. Enable CONFIG_CGROUPS
b. Enable CONFIG_RESOURCE_COUNTERS
c. Enable CONFIG_CGROUP_MEM_RES_CTLR
1. Prepare the cgroups
# mkdir -p /cgroups
# mount -t cgroup none /cgroups -o memory
2. Make the new group and move bash into it
# mkdir /cgroups/0
# echo $$ > /cgroups/0/tasks
Since now we're in the 0 cgroup,
We can alter the memory limit:
# echo 4M > /cgroups/0/memory.limit_in_bytes
NOTE: We can use a suffix (k, K, m, M, g or G) to indicate values in kilo,
mega or gigabytes.
# cat /cgroups/0/memory.limit_in_bytes
4194304
NOTE: The interface has now changed to display the usage in bytes
instead of pages
We can check the usage:
# cat /cgroups/0/memory.usage_in_bytes
1216512
A successful write to this file does not guarantee a successful set of
this limit to the value written into the file. This can be due to a
number of factors, such as rounding up to page boundaries or the total
availability of memory on the system. The user is required to re-read
this file after a write to guarantee the value committed by the kernel.
# echo 1 > memory.limit_in_bytes
# cat memory.limit_in_bytes
4096
The memory.failcnt field gives the number of times that the cgroup limit was
exceeded.
The memory.stat file gives accounting information. Now, the number of
caches, RSS and Active pages/Inactive pages are shown.
4. Testing
Balbir posted lmbench, AIM9, LTP and vmmstress results [10] and [11].
Apart from that v6 has been tested with several applications and regular
daily use. The controller has also been tested on the PPC64, x86_64 and
UML platforms.
4.1 Troubleshooting
Sometimes a user might find that the application under a cgroup is
terminated. There are several causes for this:
1. The cgroup limit is too low (just too low to do anything useful)
2. The user is using anonymous memory and swap is turned off or too low
A sync followed by echo 1 > /proc/sys/vm/drop_caches will help get rid of
some of the pages cached in the cgroup (page cache pages).
4.2 Task migration
When a task migrates from one cgroup to another, it's charge is not
carried forward. The pages allocated from the original cgroup still
remain charged to it, the charge is dropped when the page is freed or
reclaimed.
4.3 Removing a cgroup
A cgroup can be removed by rmdir, but as discussed in sections 4.1 and 4.2, a
cgroup might have some charge associated with it, even though all
tasks have migrated away from it.
Such charges are freed(at default) or moved to its parent. When moved,
both of RSS and CACHES are moved to parent.
If both of them are busy, rmdir() returns -EBUSY. See 5.1 Also.
Charges recorded in swap information is not updated at removal of cgroup.
Recorded information is discarded and a cgroup which uses swap (swapcache)
will be charged as a new owner of it.
5. Misc. interfaces.
5.1 force_empty
memory.force_empty interface is provided to make cgroup's memory usage empty.
You can use this interface only when the cgroup has no tasks.
When writing anything to this
# echo 0 > memory.force_empty
Almost all pages tracked by this memcg will be unmapped and freed. Some of
pages cannot be freed because it's locked or in-use. Such pages are moved
to parent and this cgroup will be empty. But this may return -EBUSY in
some too busy case.
Typical use case of this interface is that calling this before rmdir().
Because rmdir() moves all pages to parent, some out-of-use page caches can be
moved to the parent. If you want to avoid that, force_empty will be useful.
5.2 stat file
memory.stat file includes following statistics (now)
cache - # of pages from page-cache and shmem.
rss - # of pages from anonymous memory.
pgpgin - # of event of charging
pgpgout - # of event of uncharging
active_anon - # of pages on active lru of anon, shmem.
inactive_anon - # of pages on active lru of anon, shmem
active_file - # of pages on active lru of file-cache
inactive_file - # of pages on inactive lru of file cache
unevictable - # of pages cannot be reclaimed.(mlocked etc)
Below is depend on CONFIG_DEBUG_VM.
inactive_ratio - VM inernal parameter. (see mm/page_alloc.c)
recent_rotated_anon - VM internal parameter. (see mm/vmscan.c)
recent_rotated_file - VM internal parameter. (see mm/vmscan.c)
recent_scanned_anon - VM internal parameter. (see mm/vmscan.c)
recent_scanned_file - VM internal parameter. (see mm/vmscan.c)
Memo:
recent_rotated means recent frequency of lru rotation.
recent_scanned means recent # of scans to lru.
showing for better debug please see the code for meanings.
5.3 swappiness
Similar to /proc/sys/vm/swappiness, but affecting a hierarchy of groups only.
Following cgroup's swapiness can't be changed.
- root cgroup (uses /proc/sys/vm/swappiness).
- a cgroup which uses hierarchy and it has child cgroup.
- a cgroup which uses hierarchy and not the root of hierarchy.
6. Hierarchy support
The memory controller supports a deep hierarchy and hierarchical accounting.
The hierarchy is created by creating the appropriate cgroups in the
cgroup filesystem. Consider for example, the following cgroup filesystem
hierarchy
root
/ | \
/ | \
a b c
| \
| \
d e
In the diagram above, with hierarchical accounting enabled, all memory
usage of e, is accounted to its ancestors up until the root (i.e, c and root),
that has memory.use_hierarchy enabled. If one of the ancestors goes over its
limit, the reclaim algorithm reclaims from the tasks in the ancestor and the
children of the ancestor.
6.1 Enabling hierarchical accounting and reclaim
The memory controller by default disables the hierarchy feature. Support
can be enabled by writing 1 to memory.use_hierarchy file of the root cgroup
# echo 1 > memory.use_hierarchy
The feature can be disabled by
# echo 0 > memory.use_hierarchy
NOTE1: Enabling/disabling will fail if the cgroup already has other
cgroups created below it.
NOTE2: This feature can be enabled/disabled per subtree.
7. TODO
1. Add support for accounting huge pages (as a separate controller)
2. Make per-cgroup scanner reclaim not-shared pages first
3. Teach controller to account for shared-pages
4. Start reclamation in the background when the limit is
not yet hit but the usage is getting closer
Summary
Overall, the memory controller has been a stable controller and has been
commented and discussed quite extensively in the community.
References
1. Singh, Balbir. RFC: Memory Controller, http://lwn.net/Articles/206697/
2. Singh, Balbir. Memory Controller (RSS Control),
http://lwn.net/Articles/222762/
3. Emelianov, Pavel. Resource controllers based on process cgroups
http://lkml.org/lkml/2007/3/6/198
4. Emelianov, Pavel. RSS controller based on process cgroups (v2)
http://lkml.org/lkml/2007/4/9/78
5. Emelianov, Pavel. RSS controller based on process cgroups (v3)
http://lkml.org/lkml/2007/5/30/244
6. Menage, Paul. Control Groups v10, http://lwn.net/Articles/236032/
7. Vaidyanathan, Srinivasan, Control Groups: Pagecache accounting and control
subsystem (v3), http://lwn.net/Articles/235534/
8. Singh, Balbir. RSS controller v2 test results (lmbench),
http://lkml.org/lkml/2007/5/17/232
9. Singh, Balbir. RSS controller v2 AIM9 results
http://lkml.org/lkml/2007/5/18/1
10. Singh, Balbir. Memory controller v6 test results,
http://lkml.org/lkml/2007/8/19/36
11. Singh, Balbir. Memory controller introduction (v6),
http://lkml.org/lkml/2007/8/17/69
12. Corbet, Jonathan, Controlling memory use in cgroups,
http://lwn.net/Articles/243795/
Memory Resource Controller
NOTE: The Memory Resource Controller has been generically been referred
to as the memory controller in this document. Do not confuse memory controller
used here with the memory controller that is used in hardware.
Salient features
a. Enable control of both RSS (mapped) and Page Cache (unmapped) pages
b. The infrastructure allows easy addition of other types of memory to control
c. Provides *zero overhead* for non memory controller users
d. Provides a double LRU: global memory pressure causes reclaim from the
global LRU; a cgroup on hitting a limit, reclaims from the per
cgroup LRU
NOTE: Swap Cache (unmapped) is not accounted now.
Benefits and Purpose of the memory controller
The memory controller isolates the memory behaviour of a group of tasks
from the rest of the system. The article on LWN [12] mentions some probable
uses of the memory controller. The memory controller can be used to
a. Isolate an application or a group of applications
Memory hungry applications can be isolated and limited to a smaller
amount of memory.
b. Create a cgroup with limited amount of memory, this can be used
as a good alternative to booting with mem=XXXX.
c. Virtualization solutions can control the amount of memory they want
to assign to a virtual machine instance.
d. A CD/DVD burner could control the amount of memory used by the
rest of the system to ensure that burning does not fail due to lack
of available memory.
e. There are several other use cases, find one or use the controller just
for fun (to learn and hack on the VM subsystem).
1. History
The memory controller has a long history. A request for comments for the memory
controller was posted by Balbir Singh [1]. At the time the RFC was posted
there were several implementations for memory control. The goal of the
RFC was to build consensus and agreement for the minimal features required
for memory control. The first RSS controller was posted by Balbir Singh[2]
in Feb 2007. Pavel Emelianov [3][4][5] has since posted three versions of the
RSS controller. At OLS, at the resource management BoF, everyone suggested
that we handle both page cache and RSS together. Another request was raised
to allow user space handling of OOM. The current memory controller is
at version 6; it combines both mapped (RSS) and unmapped Page
Cache Control [11].
2. Memory Control
Memory is a unique resource in the sense that it is present in a limited
amount. If a task requires a lot of CPU processing, the task can spread
its processing over a period of hours, days, months or years, but with
memory, the same physical memory needs to be reused to accomplish the task.
The memory controller implementation has been divided into phases. These
are:
1. Memory controller
2. mlock(2) controller
3. Kernel user memory accounting and slab control
4. user mappings length controller
The memory controller is the first controller developed.
2.1. Design
The core of the design is a counter called the res_counter. The res_counter
tracks the current memory usage and limit of the group of processes associated
with the controller. Each cgroup has a memory controller specific data
structure (mem_cgroup) associated with it.
2.2. Accounting
+--------------------+
| mem_cgroup |
| (res_counter) |
+--------------------+
/ ^ \
/ | \
+---------------+ | +---------------+
| mm_struct | |.... | mm_struct |
| | | | |
+---------------+ | +---------------+
|
+ --------------+
|
+---------------+ +------+--------+
| page +----------> page_cgroup|
| | | |
+---------------+ +---------------+
(Figure 1: Hierarchy of Accounting)
Figure 1 shows the important aspects of the controller
1. Accounting happens per cgroup
2. Each mm_struct knows about which cgroup it belongs to
3. Each page has a pointer to the page_cgroup, which in turn knows the
cgroup it belongs to
The accounting is done as follows: mem_cgroup_charge() is invoked to setup
the necessary data structures and check if the cgroup that is being charged
is over its limit. If it is then reclaim is invoked on the cgroup.
More details can be found in the reclaim section of this document.
If everything goes well, a page meta-data-structure called page_cgroup is
allocated and associated with the page. This routine also adds the page to
the per cgroup LRU.
2.2.1 Accounting details
All mapped anon pages (RSS) and cache pages (Page Cache) are accounted.
(some pages which never be reclaimable and will not be on global LRU
are not accounted. we just accounts pages under usual vm management.)
RSS pages are accounted at page_fault unless they've already been accounted
for earlier. A file page will be accounted for as Page Cache when it's
inserted into inode (radix-tree). While it's mapped into the page tables of
processes, duplicate accounting is carefully avoided.
A RSS page is unaccounted when it's fully unmapped. A PageCache page is
unaccounted when it's removed from radix-tree.
At page migration, accounting information is kept.
Note: we just account pages-on-lru because our purpose is to control amount
of used pages. not-on-lru pages are tend to be out-of-control from vm view.
2.3 Shared Page Accounting
Shared pages are accounted on the basis of the first touch approach. The
cgroup that first touches a page is accounted for the page. The principle
behind this approach is that a cgroup that aggressively uses a shared
page will eventually get charged for it (once it is uncharged from
the cgroup that brought it in -- this will happen on memory pressure).
2.4 Reclaim
Each cgroup maintains a per cgroup LRU that consists of an active
and inactive list. When a cgroup goes over its limit, we first try
to reclaim memory from the cgroup so as to make space for the new
pages that the cgroup has touched. If the reclaim is unsuccessful,
an OOM routine is invoked to select and kill the bulkiest task in the
cgroup.
The reclaim algorithm has not been modified for cgroups, except that
pages that are selected for reclaiming come from the per cgroup LRU
list.
2. Locking
The memory controller uses the following hierarchy
1. zone->lru_lock is used for selecting pages to be isolated
2. mem->per_zone->lru_lock protects the per cgroup LRU (per zone)
3. lock_page_cgroup() is used to protect page->page_cgroup
3. User Interface
0. Configuration
a. Enable CONFIG_CGROUPS
b. Enable CONFIG_RESOURCE_COUNTERS
c. Enable CONFIG_CGROUP_MEM_RES_CTLR
1. Prepare the cgroups
# mkdir -p /cgroups
# mount -t cgroup none /cgroups -o memory
2. Make the new group and move bash into it
# mkdir /cgroups/0
# echo $$ > /cgroups/0/tasks
Since now we're in the 0 cgroup,
We can alter the memory limit:
# echo 4M > /cgroups/0/memory.limit_in_bytes
NOTE: We can use a suffix (k, K, m, M, g or G) to indicate values in kilo,
mega or gigabytes.
# cat /cgroups/0/memory.limit_in_bytes
4194304
NOTE: The interface has now changed to display the usage in bytes
instead of pages
We can check the usage:
# cat /cgroups/0/memory.usage_in_bytes
1216512
A successful write to this file does not guarantee a successful set of
this limit to the value written into the file. This can be due to a
number of factors, such as rounding up to page boundaries or the total
availability of memory on the system. The user is required to re-read
this file after a write to guarantee the value committed by the kernel.
# echo 1 > memory.limit_in_bytes
# cat memory.limit_in_bytes
4096
The memory.failcnt field gives the number of times that the cgroup limit was
exceeded.
The memory.stat file gives accounting information. Now, the number of
caches, RSS and Active pages/Inactive pages are shown.
The memory.force_empty gives an interface to drop *all* charges by force.
# echo 1 > memory.force_empty
will drop all charges in cgroup. Currently, this is maintained for test.
4. Testing
Balbir posted lmbench, AIM9, LTP and vmmstress results [10] and [11].
Apart from that v6 has been tested with several applications and regular
daily use. The controller has also been tested on the PPC64, x86_64 and
UML platforms.
4.1 Troubleshooting
Sometimes a user might find that the application under a cgroup is
terminated. There are several causes for this:
1. The cgroup limit is too low (just too low to do anything useful)
2. The user is using anonymous memory and swap is turned off or too low
A sync followed by echo 1 > /proc/sys/vm/drop_caches will help get rid of
some of the pages cached in the cgroup (page cache pages).
4.2 Task migration
When a task migrates from one cgroup to another, it's charge is not
carried forward. The pages allocated from the original cgroup still
remain charged to it, the charge is dropped when the page is freed or
reclaimed.
4.3 Removing a cgroup
A cgroup can be removed by rmdir, but as discussed in sections 4.1 and 4.2, a
cgroup might have some charge associated with it, even though all
tasks have migrated away from it. Such charges are automatically dropped at
rmdir() if there are no tasks.
5. TODO
1. Add support for accounting huge pages (as a separate controller)
2. Make per-cgroup scanner reclaim not-shared pages first
3. Teach controller to account for shared-pages
4. Start reclamation in the background when the limit is
not yet hit but the usage is getting closer
Summary
Overall, the memory controller has been a stable controller and has been
commented and discussed quite extensively in the community.
References
1. Singh, Balbir. RFC: Memory Controller, http://lwn.net/Articles/206697/
2. Singh, Balbir. Memory Controller (RSS Control),
http://lwn.net/Articles/222762/
3. Emelianov, Pavel. Resource controllers based on process cgroups
http://lkml.org/lkml/2007/3/6/198
4. Emelianov, Pavel. RSS controller based on process cgroups (v2)
http://lkml.org/lkml/2007/4/9/78
5. Emelianov, Pavel. RSS controller based on process cgroups (v3)
http://lkml.org/lkml/2007/5/30/244
6. Menage, Paul. Control Groups v10, http://lwn.net/Articles/236032/
7. Vaidyanathan, Srinivasan, Control Groups: Pagecache accounting and control
subsystem (v3), http://lwn.net/Articles/235534/
8. Singh, Balbir. RSS controller v2 test results (lmbench),
http://lkml.org/lkml/2007/5/17/232
9. Singh, Balbir. RSS controller v2 AIM9 results
http://lkml.org/lkml/2007/5/18/1
10. Singh, Balbir. Memory controller v6 test results,
http://lkml.org/lkml/2007/8/19/36
11. Singh, Balbir. Memory controller introduction (v6),
http://lkml.org/lkml/2007/8/17/69
12. Corbet, Jonathan, Controlling memory use in cgroups,
http://lwn.net/Articles/243795/
......@@ -93,10 +93,8 @@ Several "PowerBook" and "iBook2" notebooks are supported.
1.5 SuperH
----------
The following SuperH processors are supported by cpufreq:
SH-3
SH-4
All SuperH processors supporting rate rounding through the clock
framework are supported by cpufreq.
1.6 Blackfin
------------
......
......@@ -50,16 +50,17 @@ additional_cpus=n (*) Use this to limit hotpluggable cpus. This option sets
cpu_possible_map = cpu_present_map + additional_cpus
(*) Option valid only for following architectures
- x86_64, ia64
- ia64
ia64 and x86_64 use the number of disabled local apics in ACPI tables MADT
to determine the number of potentially hot-pluggable cpus. The implementation
should only rely on this to count the # of cpus, but *MUST* not rely on the
apicid values in those tables for disabled apics. In the event BIOS doesn't
mark such hot-pluggable cpus as disabled entries, one could use this
parameter "additional_cpus=x" to represent those cpus in the cpu_possible_map.
ia64 uses the number of disabled local apics in ACPI tables MADT to
determine the number of potentially hot-pluggable cpus. The implementation
should only rely on this to count the # of cpus, but *MUST* not rely
on the apicid values in those tables for disabled apics. In the event
BIOS doesn't mark such hot-pluggable cpus as disabled entries, one could
use this parameter "additional_cpus=x" to represent those cpus in the
cpu_possible_map.
possible_cpus=n [s390 only] use this to set hotpluggable cpus.
possible_cpus=n [s390,x86_64] use this to set hotpluggable cpus.
This option sets possible_cpus bits in
cpu_possible_map. Thus keeping the numbers of bits set
constant even if the machine gets rebooted.
......
......@@ -31,3 +31,51 @@ not defined by include/asm-XXX/topology.h:
2) core_id: 0
3) thread_siblings: just the given CPU
4) core_siblings: just the given CPU
Additionally, cpu topology information is provided under
/sys/devices/system/cpu and includes these files. The internal
source for the output is in brackets ("[]").
kernel_max: the maximum cpu index allowed by the kernel configuration.
[NR_CPUS-1]
offline: cpus that are not online because they have been
HOTPLUGGED off (see cpu-hotplug.txt) or exceed the limit
of cpus allowed by the kernel configuration (kernel_max
above). [~cpu_online_mask + cpus >= NR_CPUS]
online: cpus that are online and being scheduled [cpu_online_mask]
possible: cpus that have been allocated resources and can be
brought online if they are present. [cpu_possible_mask]
present: cpus that have been identified as being present in the
system. [cpu_present_mask]
The format for the above output is compatible with cpulist_parse()
[see <linux/cpumask.h>]. Some examples follow.
In this example, there are 64 cpus in the system but cpus 32-63 exceed
the kernel max which is limited to 0..31 by the NR_CPUS config option
being 32. Note also that cpus 2 and 4-31 are not online but could be
brought online as they are both present and possible.
kernel_max: 31
offline: 2,4-31,32-63
online: 0-1,3
possible: 0-31
present: 0-31
In this example, the NR_CPUS config option is 128, but the kernel was
started with possible_cpus=144. There are 4 cpus in the system and cpu2
was manually taken offline (and is the only cpu that can be brought
online.)
kernel_max: 127
offline: 2,4-127,128-143
online: 0-1,3
possible: 0-127
present: 0-3
See cpu-hotplug.txt for the possible_cpus=NUM kernel start parameter
as well as more information on the various cpumask's.
此差异已折叠。
......@@ -13,9 +13,9 @@
3.6 Constraints
3.7 Example
4 DRIVER DEVELOPER NOTES
4 DMAENGINE DRIVER DEVELOPER NOTES
4.1 Conformance points
4.2 "My application needs finer control of hardware channels"
4.2 "My application needs exclusive control of hardware channels"
5 SOURCE
......@@ -150,6 +150,7 @@ ops_run_* and ops_complete_* routines in drivers/md/raid5.c for more
implementation examples.
4 DRIVER DEVELOPMENT NOTES
4.1 Conformance points:
There are a few conformance points required in dmaengine drivers to
accommodate assumptions made by applications using the async_tx API:
......@@ -158,58 +159,49 @@ accommodate assumptions made by applications using the async_tx API:
3/ Use async_tx_run_dependencies() in the descriptor clean up path to
handle submission of dependent operations
4.2 "My application needs finer control of hardware channels"
This requirement seems to arise from cases where a DMA engine driver is
trying to support device-to-memory DMA. The dmaengine and async_tx
implementations were designed for offloading memory-to-memory
operations; however, there are some capabilities of the dmaengine layer
that can be used for platform-specific channel management.
Platform-specific constraints can be handled by registering the
application as a 'dma_client' and implementing a 'dma_event_callback' to
apply a filter to the available channels in the system. Before showing
how to implement a custom dma_event callback some background of
dmaengine's client support is required.
The following routines in dmaengine support multiple clients requesting
use of a channel:
- dma_async_client_register(struct dma_client *client)
- dma_async_client_chan_request(struct dma_client *client)
dma_async_client_register takes a pointer to an initialized dma_client
structure. It expects that the 'event_callback' and 'cap_mask' fields
are already initialized.
dma_async_client_chan_request triggers dmaengine to notify the client of
all channels that satisfy the capability mask. It is up to the client's
event_callback routine to track how many channels the client needs and
how many it is currently using. The dma_event_callback routine returns a
dma_state_client code to let dmaengine know the status of the
allocation.
Below is the example of how to extend this functionality for
platform-specific filtering of the available channels beyond the
standard capability mask:
static enum dma_state_client
my_dma_client_callback(struct dma_client *client,
struct dma_chan *chan, enum dma_state state)
{
struct dma_device *dma_dev;
struct my_platform_specific_dma *plat_dma_dev;
dma_dev = chan->device;
plat_dma_dev = container_of(dma_dev,
struct my_platform_specific_dma,
dma_dev);
if (!plat_dma_dev->platform_specific_capability)
return DMA_DUP;
. . .
}
4.2 "My application needs exclusive control of hardware channels"
Primarily this requirement arises from cases where a DMA engine driver
is being used to support device-to-memory operations. A channel that is
performing these operations cannot, for many platform specific reasons,
be shared. For these cases the dma_request_channel() interface is
provided.
The interface is:
struct dma_chan *dma_request_channel(dma_cap_mask_t mask,
dma_filter_fn filter_fn,
void *filter_param);
Where dma_filter_fn is defined as:
typedef bool (*dma_filter_fn)(struct dma_chan *chan, void *filter_param);
When the optional 'filter_fn' parameter is set to NULL
dma_request_channel simply returns the first channel that satisfies the
capability mask. Otherwise, when the mask parameter is insufficient for
specifying the necessary channel, the filter_fn routine can be used to
disposition the available channels in the system. The filter_fn routine
is called once for each free channel in the system. Upon seeing a
suitable channel filter_fn returns DMA_ACK which flags that channel to
be the return value from dma_request_channel. A channel allocated via
this interface is exclusive to the caller, until dma_release_channel()
is called.
The DMA_PRIVATE capability flag is used to tag dma devices that should
not be used by the general-purpose allocator. It can be set at
initialization time if it is known that a channel will always be
private. Alternatively, it is set when dma_request_channel() finds an
unused "public" channel.
A couple caveats to note when implementing a driver and consumer:
1/ Once a channel has been privately allocated it will no longer be
considered by the general-purpose allocator even after a call to
dma_release_channel().
2/ Since capabilities are specified at the device level a dma_device
with multiple channels will either have all channels public, or all
channels private.
5 SOURCE
include/linux/dmaengine.h: core header file for DMA drivers and clients
include/linux/dmaengine.h: core header file for DMA drivers and api users
drivers/dma/dmaengine.c: offload engine channel management routines
drivers/dma/: location for offload engine drivers
include/linux/async_tx.h: core header file for the async_tx api
......
......@@ -81,8 +81,8 @@ Until this step is completed the driver cannot be unloaded.
Also echoing either mono ,packet or init in to image_type will free up the
memory allocated by the driver.
If an user by accident executes steps 1 and 3 above without executing step 2;
it will make the /sys/class/firmware/dell_rbu/ entries to disappear.
If a user by accident executes steps 1 and 3 above without executing step 2;
it will make the /sys/class/firmware/dell_rbu/ entries disappear.
The entries can be recreated by doing the following
echo init > /sys/devices/platform/dell_rbu/image_type
NOTE: echoing init in image_type does not change it original value.
......
......@@ -375,10 +375,10 @@ say, this can be a large job, so it is best to be sure that the
justification is solid.
When making an incompatible API change, one should, whenever possible,
ensure that code which has not been updated is caught by the compiler.
ensure that code which has not been updated is caught by the compiler.
This will help you to be sure that you have found all in-tree uses of that
interface. It will also alert developers of out-of-tree code that there is
a change that they need to respond to. Supporting out-of-tree code is not
something that kernel developers need to be worried about, but we also do
not have to make life harder for out-of-tree developers than it it needs to
be.
not have to make life harder for out-of-tree developers than it needs to
be.
See Documentation/crypto/async-tx-api.txt
How to set up the Technisat devices
===================================
1) Find out what device you have
================================
First start your linux box with a shipped kernel:
lspci -vvv for a PCI device (lsusb -vvv for an USB device) will show you for example:
02:0b.0 Network controller: Techsan Electronics Co Ltd B2C2 FlexCopII DVB chip / Technisat SkyStar2 DVB card (rev 02)
dmesg | grep frontend may show you for example:
DVB: registering frontend 0 (Conexant CX24123/CX24109)...
2) Kernel compilation:
======================
If the Technisat is the only TV device in your box get rid of unnecessary modules and check this one:
"Multimedia devices" => "Customise analog and hybrid tuner modules to build"
In this directory uncheck every driver which is activated there.
Then please activate:
2a) Main module part:
a.)"Multimedia devices" => "DVB/ATSC adapters" => "Technisat/B2C2 FlexcopII(b) and FlexCopIII adapters"
b.)"Multimedia devices" => "DVB/ATSC adapters" => "Technisat/B2C2 FlexcopII(b) and FlexCopIII adapters" => "Technisat/B2C2 Air/Sky/Cable2PC PCI" in case of a PCI card OR
c.)"Multimedia devices" => "DVB/ATSC adapters" => "Technisat/B2C2 FlexcopII(b) and FlexCopIII adapters" => "Technisat/B2C2 Air/Sky/Cable2PC USB" in case of an USB 1.1 adapter
d.)"Multimedia devices" => "DVB/ATSC adapters" => "Technisat/B2C2 FlexcopII(b) and FlexCopIII adapters" => "Enable debug for the B2C2 FlexCop drivers"
Notice: d.) is helpful for troubleshooting
2b) Frontend module part:
1.) Revision 2.3:
a.)"Multimedia devices" => "Customise DVB frontends" => "Customise the frontend modules to build"
b.)"Multimedia devices" => "Customise DVB frontends" => "Zarlink VP310/MT312/ZL10313 based"
2.) Revision 2.6:
a.)"Multimedia devices" => "Customise DVB frontends" => "Customise the frontend modules to build"
b.)"Multimedia devices" => "Customise DVB frontends" => "ST STV0299 based"
3.) Revision 2.7:
a.)"Multimedia devices" => "Customise DVB frontends" => "Customise the frontend modules to build"
b.)"Multimedia devices" => "Customise DVB frontends" => "Samsung S5H1420 based"
c.)"Multimedia devices" => "Customise DVB frontends" => "Integrant ITD1000 Zero IF tuner for DVB-S/DSS"
d.)"Multimedia devices" => "Customise DVB frontends" => "ISL6421 SEC controller"
4.) Revision 2.8:
a.)"Multimedia devices" => "Customise DVB frontends" => "Customise the frontend modules to build"
b.)"Multimedia devices" => "Customise DVB frontends" => "Conexant CX24113/CX24128 tuner for DVB-S/DSS"
c.)"Multimedia devices" => "Customise DVB frontends" => "Conexant CX24123 based"
d.)"Multimedia devices" => "Customise DVB frontends" => "ISL6421 SEC controller"
5.) DVB-T card:
a.)"Multimedia devices" => "Customise DVB frontends" => "Customise the frontend modules to build"
b.)"Multimedia devices" => "Customise DVB frontends" => "Zarlink MT352 based"
6.) DVB-C card:
a.)"Multimedia devices" => "Customise DVB frontends" => "Customise the frontend modules to build"
b.)"Multimedia devices" => "Customise DVB frontends" => "ST STV0297 based"
7.) ATSC card 1st generation:
a.)"Multimedia devices" => "Customise DVB frontends" => "Customise the frontend modules to build"
b.)"Multimedia devices" => "Customise DVB frontends" => "Broadcom BCM3510"
8.) ATSC card 2nd generation:
a.)"Multimedia devices" => "Customise DVB frontends" => "Customise the frontend modules to build"
b.)"Multimedia devices" => "Customise DVB frontends" => "NxtWave Communications NXT2002/NXT2004 based"
c.)"Multimedia devices" => "Customise DVB frontends" => "LG Electronics LGDT3302/LGDT3303 based"
Author: Uwe Bugla <uwe.bugla@gmx.de> December 2008
......@@ -5,9 +5,13 @@ The driver supports the following options, either via
options=<OPTIONS> when modular or video=pxafb:<OPTIONS> when built in.
For example:
modprobe pxafb options=mode:640x480-8,passive
modprobe pxafb options=vmem:2M,mode:640x480-8,passive
or on the kernel command line
video=pxafb:mode:640x480-8,passive
video=pxafb:vmem:2M,mode:640x480-8,passive
vmem: VIDEO_MEM_SIZE
Amount of video memory to allocate (can be suffixed with K or M
for kilobytes or megabytes)
mode:XRESxYRES[-BPP]
XRES == LCCR1_PPL + 1
......@@ -52,3 +56,87 @@ outputen:POLARITY
pixclockpol:POLARITY
pixel clock polarity
0 => falling edge, 1 => rising edge
Overlay Support for PXA27x and later LCD controllers
====================================================
PXA27x and later processors support overlay1 and overlay2 on-top of the
base framebuffer (although under-neath the base is also possible). They
support palette and no-palette RGB formats, as well as YUV formats (only
available on overlay2). These overlays have dedicated DMA channels and
behave in a similar way as a framebuffer.
However, there are some differences between these overlay framebuffers
and normal framebuffers, as listed below:
1. overlay can start at a 32-bit word aligned position within the base
framebuffer, which means they have a start (x, y). This information
is encoded into var->nonstd (no, var->xoffset and var->yoffset are
not for such purpose).
2. overlay framebuffer is allocated dynamically according to specified
'struct fb_var_screeninfo', the amount is decided by:
var->xres_virtual * var->yres_virtual * bpp
bpp = 16 -- for RGB565 or RGBT555
= 24 -- for YUV444 packed
= 24 -- for YUV444 planar
= 16 -- for YUV422 planar (1 pixel = 1 Y + 1/2 Cb + 1/2 Cr)
= 12 -- for YUV420 planar (1 pixel = 1 Y + 1/4 Cb + 1/4 Cr)
NOTE:
a. overlay does not support panning in x-direction, thus
var->xres_virtual will always be equal to var->xres
b. line length of overlay(s) must be on a 32-bit word boundary,
for YUV planar modes, it is a requirement for the component
with minimum bits per pixel, e.g. for YUV420, Cr component
for one pixel is actually 2-bits, it means the line length
should be a multiple of 16-pixels
c. starting horizontal position (XPOS) should start on a 32-bit
word boundary, otherwise the fb_check_var() will just fail.
d. the rectangle of the overlay should be within the base plane,
otherwise fail
Applications should follow the sequence below to operate an overlay
framebuffer:
a. open("/dev/fb[1-2]", ...)
b. ioctl(fd, FBIOGET_VSCREENINFO, ...)
c. modify 'var' with desired parameters:
1) var->xres and var->yres
2) larger var->yres_virtual if more memory is required,
usually for double-buffering
3) var->nonstd for starting (x, y) and color format
4) var->{red, green, blue, transp} if RGB mode is to be used
d. ioctl(fd, FBIOPUT_VSCREENINFO, ...)
e. ioctl(fd, FBIOGET_FSCREENINFO, ...)
f. mmap
g. ...
3. for YUV planar formats, these are actually not supported within the
framebuffer framework, application has to take care of the offsets
and lengths of each component within the framebuffer.
4. var->nonstd is used to pass starting (x, y) position and color format,
the detailed bit fields are shown below:
31 23 20 10 0
+-----------------+---+----------+----------+
| ... unused ... |FOR| XPOS | YPOS |
+-----------------+---+----------+----------+
FOR - color format, as defined by OVERLAY_FORMAT_* in pxafb.h
0 - RGB
1 - YUV444 PACKED
2 - YUV444 PLANAR
3 - YUV422 PLANAR
4 - YUR420 PLANAR
XPOS - starting horizontal position
YPOS - starting vertical position
......@@ -120,13 +120,6 @@ Who: Christoph Hellwig <hch@lst.de>
---------------------------
What: eepro100 network driver
When: January 2007
Why: replaced by the e100 driver
Who: Adrian Bunk <bunk@stusta.de>
---------------------------
What: Unused EXPORT_SYMBOL/EXPORT_SYMBOL_GPL exports
(temporary transition config option provided until then)
The transition config option will also be removed at the same time.
......@@ -244,18 +237,6 @@ Who: Michael Buesch <mb@bu3sch.de>
---------------------------
What: init_mm export
When: 2.6.26
Why: Not used in-tree. The current out-of-tree users used it to
work around problems in the CPA code which should be resolved
by now. One usecase was described to provide verification code
of the CPA operation. That's a good idea in general, but such
code / infrastructure should be in the kernel and not in some
out-of-tree driver.
Who: Thomas Gleixner <tglx@linutronix.de>
----------------------------
What: usedac i386 kernel parameter
When: 2.6.27
Why: replaced by allowdac and no dac combination
......@@ -329,17 +310,28 @@ Who: Krzysztof Piotr Oledzki <ole@ans.pl>
---------------------------
What: ide-scsi (BLK_DEV_IDESCSI)
When: 2.6.29
Why: The 2.6 kernel supports direct writing to ide CD drives, which
eliminates the need for ide-scsi. The new method is more
efficient in every way.
Who: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
---------------------------
What: i2c_attach_client(), i2c_detach_client(), i2c_driver->detach_client()
When: 2.6.29 (ideally) or 2.6.30 (more likely)
Why: Deprecated by the new (standard) device driver binding model. Use
i2c_driver->probe() and ->remove() instead.
Who: Jean Delvare <khali@linux-fr.org>
---------------------------
What: fscher and fscpos drivers
When: June 2009
Why: Deprecated by the new fschmd driver.
Who: Hans de Goede <hdegoede@redhat.com>
Jean Delvare <khali@linux-fr.org>
---------------------------
What: SELinux "compat_net" functionality
When: 2.6.30 at the earliest
Why: In 2.6.18 the Secmark concept was introduced to replace the "compat_net"
network access control functionality of SELinux. Secmark offers both
better performance and greater flexibility than the "compat_net"
mechanism. Now that the major Linux distributions have moved to
Secmark, it is time to deprecate the older mechanism and start the
process of removing the old code.
Who: Paul Moore <paul.moore@hp.com>
......@@ -97,8 +97,8 @@ prototypes:
void (*put_super) (struct super_block *);
void (*write_super) (struct super_block *);
int (*sync_fs)(struct super_block *sb, int wait);
void (*write_super_lockfs) (struct super_block *);
void (*unlockfs) (struct super_block *);
int (*freeze_fs) (struct super_block *);
int (*unfreeze_fs) (struct super_block *);
int (*statfs) (struct dentry *, struct kstatfs *);
int (*remount_fs) (struct super_block *, int *, char *);
void (*clear_inode) (struct inode *);
......@@ -119,8 +119,8 @@ delete_inode: no
put_super: yes yes no
write_super: no yes read
sync_fs: no no read
write_super_lockfs: ?
unlockfs: ?
freeze_fs: ?
unfreeze_fs: ?
statfs: no no no
remount_fs: yes yes maybe (see below)
clear_inode: no
......@@ -394,11 +394,10 @@ prototypes:
unsigned long (*get_unmapped_area)(struct file *, unsigned long,
unsigned long, unsigned long, unsigned long);
int (*check_flags)(int);
int (*dir_notify)(struct file *, unsigned long);
};
locking rules:
All except ->poll() may block.
All may block.
BKL
llseek: no (see below)
read: no
......@@ -424,7 +423,6 @@ sendfile: no
sendpage: no
get_unmapped_area: no
check_flags: no
dir_notify: no
->llseek() locking has moved from llseek to the individual llseek
implementations. If your fs is not using generic_file_llseek, you
......
BTRFS
=====
Btrfs is a new copy on write filesystem for Linux aimed at
implementing advanced features while focusing on fault tolerance,
repair and easy administration. Initially developed by Oracle, Btrfs
is licensed under the GPL and open for contribution from anyone.
Linux has a wealth of filesystems to choose from, but we are facing a
number of challenges with scaling to the large storage subsystems that
are becoming common in today's data centers. Filesystems need to scale
in their ability to address and manage large storage, and also in
their ability to detect, repair and tolerate errors in the data stored
on disk. Btrfs is under heavy development, and is not suitable for
any uses other than benchmarking and review. The Btrfs disk format is
not yet finalized.
The main Btrfs features include:
* Extent based file storage (2^64 max file size)
* Space efficient packing of small files
* Space efficient indexed directories
* Dynamic inode allocation
* Writable snapshots
* Subvolumes (separate internal filesystem roots)
* Object level mirroring and striping
* Checksums on data and metadata (multiple algorithms available)
* Compression
* Integrated multiple device support, with several raid algorithms
* Online filesystem check (not yet implemented)
* Very fast offline filesystem check
* Efficient incremental backup and FS mirroring (not yet implemented)
* Online filesystem defragmentation
MAILING LIST
============
There is a Btrfs mailing list hosted on vger.kernel.org. You can
find details on how to subscribe here:
http://vger.kernel.org/vger-lists.html#linux-btrfs
Mailing list archives are available from gmane:
http://dir.gmane.org/gmane.comp.file-systems.btrfs
IRC
===
Discussion of Btrfs also occurs on the #btrfs channel of the Freenode
IRC network.
UTILITIES
=========
Userspace tools for creating and manipulating Btrfs file systems are
available from the git repository at the following location:
http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-progs-unstable.git
git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git
These include the following tools:
mkfs.btrfs: create a filesystem
btrfsctl: control program to create snapshots and subvolumes:
mount /dev/sda2 /mnt
btrfsctl -s new_subvol_name /mnt
btrfsctl -s snapshot_of_default /mnt/default
btrfsctl -s snapshot_of_new_subvol /mnt/new_subvol_name
btrfsctl -s snapshot_of_a_snapshot /mnt/snapshot_of_new_subvol
ls /mnt
default snapshot_of_a_snapshot snapshot_of_new_subvol
new_subvol_name snapshot_of_default
Snapshots and subvolumes cannot be deleted right now, but you can
rm -rf all the files and directories inside them.
btrfsck: do a limited check of the FS extent trees.
btrfs-debug-tree: print all of the FS metadata in text form. Example:
btrfs-debug-tree /dev/sda2 >& big_output_file
To support containers, we now allow multiple instances of devpts filesystem,
such that indices of ptys allocated in one instance are independent of indices
allocated in other instances of devpts.
To preserve backward compatibility, this support for multiple instances is
enabled only if:
- CONFIG_DEVPTS_MULTIPLE_INSTANCES=y, and
- '-o newinstance' mount option is specified while mounting devpts
IOW, devpts now supports both single-instance and multi-instance semantics.
If CONFIG_DEVPTS_MULTIPLE_INSTANCES=n, there is no change in behavior and
this referred to as the "legacy" mode. In this mode, the new mount options
(-o newinstance and -o ptmxmode) will be ignored with a 'bogus option' message
on console.
If CONFIG_DEVPTS_MULTIPLE_INSTANCES=y and devpts is mounted without the
'newinstance' option (as in current start-up scripts) the new mount binds
to the initial kernel mount of devpts. This mode is referred to as the
'single-instance' mode and the current, single-instance semantics are
preserved, i.e PTYs are common across the system.
The only difference between this single-instance mode and the legacy mode
is the presence of new, '/dev/pts/ptmx' node with permissions 0000, which
can safely be ignored.
If CONFIG_DEVPTS_MULTIPLE_INSTANCES=y and 'newinstance' option is specified,
the mount is considered to be in the multi-instance mode and a new instance
of the devpts fs is created. Any ptys created in this instance are independent
of ptys in other instances of devpts. Like in the single-instance mode, the
/dev/pts/ptmx node is present. To effectively use the multi-instance mode,
open of /dev/ptmx must be a redirected to '/dev/pts/ptmx' using a symlink or
bind-mount.
Eg: A container startup script could do the following:
$ chmod 0666 /dev/pts/ptmx
$ rm /dev/ptmx
$ ln -s pts/ptmx /dev/ptmx
$ ns_exec -cm /bin/bash
# We are now in new container
$ umount /dev/pts
$ mount -t devpts -o newinstance lxcpts /dev/pts
$ sshd -p 1234
where 'ns_exec -cm /bin/bash' calls clone() with CLONE_NEWNS flag and execs
/bin/bash in the child process. A pty created by the sshd is not visible in
the original mount of /dev/pts.
User-space changes
------------------
In multi-instance mode (i.e '-o newinstance' mount option is specified at least
once), following user-space issues should be noted.
1. If -o newinstance mount option is never used, /dev/pts/ptmx can be ignored
and no change is needed to system-startup scripts.
2. To effectively use multi-instance mode (i.e -o newinstance is specified)
administrators or startup scripts should "redirect" open of /dev/ptmx to
/dev/pts/ptmx using either a bind mount or symlink.
$ mount -t devpts -o newinstance devpts /dev/pts
followed by either
$ rm /dev/ptmx
$ ln -s pts/ptmx /dev/ptmx
$ chmod 666 /dev/pts/ptmx
or
$ mount -o bind /dev/pts/ptmx /dev/ptmx
3. The '/dev/ptmx -> pts/ptmx' symlink is the preferred method since it
enables better error-reporting and treats both single-instance and
multi-instance mounts similarly.
But this method requires that system-startup scripts set the mode of
/dev/pts/ptmx correctly (default mode is 0000). The scripts can set the
mode by, either
- adding ptmxmode mount option to devpts entry in /etc/fstab, or
- using 'chmod 0666 /dev/pts/ptmx'
4. If multi-instance mode mount is needed for containers, but the system
startup scripts have not yet been updated, container-startup scripts
should bind mount /dev/ptmx to /dev/pts/ptmx to avoid breaking single-
instance mounts.
Or, in general, container-startup scripts should use:
mount -t devpts -o newinstance -o ptmxmode=0666 devpts /dev/pts
if [ ! -L /dev/ptmx ]; then
mount -o bind /dev/pts/ptmx /dev/ptmx
fi
When all devpts mounts are multi-instance, /dev/ptmx can permanently be
a symlink to pts/ptmx and the bind mount can be ignored.
5. A multi-instance mount that is not accompanied by the /dev/ptmx to
/dev/pts/ptmx redirection would result in an unusable/unreachable pty.
mount -t devpts -o newinstance lxcpts /dev/pts
immediately followed by:
open("/dev/ptmx")
would create a pty, say /dev/pts/7, in the initial kernel mount.
But /dev/pts/7 would be invisible in the new mount.
6. The permissions for /dev/pts/ptmx node should be specified when mounting
/dev/pts, using the '-o ptmxmode=%o' mount option (default is 0000).
mount -t devpts -o newinstance -o ptmxmode=0644 devpts /dev/pts
The permissions can be later be changed as usual with 'chmod'.
chmod 666 /dev/pts/ptmx
7. A mount of devpts without the 'newinstance' option results in binding to
initial kernel mount. This behavior while preserving legacy semantics,
does not provide strict isolation in a container environment. i.e by
mounting devpts without the 'newinstance' option, a container could
get visibility into the 'host' or root container's devpts.
To workaround this and have strict isolation, all mounts of devpts,
including the mount in the root container, should use the newinstance
option.
......@@ -58,13 +58,22 @@ Note: More extensive information for getting started with ext4 can be
# mount -t ext4 /dev/hda1 /wherever
- When comparing performance with other filesystems, remember that
ext3/4 by default offers higher data integrity guarantees than most.
So when comparing with a metadata-only journalling filesystem, such
as ext3, use `mount -o data=writeback'. And you might as well use
`mount -o nobh' too along with it. Making the journal larger than
the mke2fs default often helps performance with metadata-intensive
workloads.
- When comparing performance with other filesystems, it's always
important to try multiple workloads; very often a subtle change in a
workload parameter can completely change the ranking of which
filesystems do well compared to others. When comparing versus ext3,
note that ext4 enables write barriers by default, while ext3 does
not enable write barriers by default. So it is useful to use
explicitly specify whether barriers are enabled or not when via the
'-o barriers=[0|1]' mount option for both ext3 and ext4 filesystems
for a fair comparison. When tuning ext3 for best benchmark numbers,
it is often worthwhile to try changing the data journaling mode; '-o
data=writeback,nobh' can be faster for some workloads. (Note
however that running mounted with data=writeback can potentially
leave stale data exposed in recently written files in case of an
unclean shutdown, which could be a security exposure in some
situations.) Configuring the filesystem with a large journal can
also be helpful for metadata-intensive workloads.
2. Features
===========
......@@ -74,7 +83,7 @@ Note: More extensive information for getting started with ext4 can be
* ability to use filesystems > 16TB (e2fsprogs support not available yet)
* extent format reduces metadata overhead (RAM, IO for access, transactions)
* extent format more robust in face of on-disk corruption due to magics,
* internal redunancy in tree
* internal redundancy in tree
* improved file allocation (multi-block alloc)
* fix 32000 subdirectory limit
* nsec timestamps for mtime, atime, ctime, create time
......@@ -116,10 +125,11 @@ grouping of bitmaps and inode tables. Some test results available here:
When mounting an ext4 filesystem, the following option are accepted:
(*) == default
extents (*) ext4 will use extents to address file data. The
file system will no longer be mountable by ext3.
noextents ext4 will not use extents for newly created files
ro Mount filesystem read only. Note that ext4 will
replay the journal (and thus write to the
partition) even when mounted "read only". The
mount options "ro,noload" can be used to prevent
writes to the filesystem.
journal_checksum Enable checksumming of the journal transactions.
This will allow the recovery code in e2fsck and the
......@@ -134,17 +144,17 @@ journal_async_commit Commit block can be written to disk without waiting
journal=update Update the ext4 file system's journal to the current
format.
journal=inum When a journal already exists, this option is ignored.
Otherwise, it specifies the number of the inode which
will represent the ext4 file system's journal file.
journal_dev=devnum When the external journal device's major/minor numbers
have changed, this option allows the user to specify
the new journal location. The journal device is
identified through its new major/minor numbers encoded
in devnum.
noload Don't load the journal on mounting.
noload Don't load the journal on mounting. Note that
if the filesystem was not unmounted cleanly,
skipping the journal replay will lead to the
filesystem containing inconsistencies that can
lead to any number of problems.
data=journal All data are committed into the journal prior to being
written into the main file system.
......@@ -219,9 +229,12 @@ minixdf Make 'df' act like Minix.
debug Extra debugging information is sent to syslog.
errors=remount-ro(*) Remount the filesystem read-only on an error.
errors=remount-ro Remount the filesystem read-only on an error.
errors=continue Keep going on a filesystem error.
errors=panic Panic and halt the machine if an error occurs.
(These mount options override the errors behavior
specified in the superblock, which can be configured
using tune2fs)
data_err=ignore(*) Just print an error message if an error occurs
in a file data buffer in ordered mode.
......@@ -261,6 +274,42 @@ delalloc (*) Deferring block allocation until write-out time.
nodelalloc Disable delayed allocation. Blocks are allocation
when data is copied from user to page cache.
max_batch_time=usec Maximum amount of time ext4 should wait for
additional filesystem operations to be batch
together with a synchronous write operation.
Since a synchronous write operation is going to
force a commit and then a wait for the I/O
complete, it doesn't cost much, and can be a
huge throughput win, we wait for a small amount
of time to see if any other transactions can
piggyback on the synchronous write. The
algorithm used is designed to automatically tune
for the speed of the disk, by measuring the
amount of time (on average) that it takes to
finish committing a transaction. Call this time
the "commit time". If the time that the
transactoin has been running is less than the
commit time, ext4 will try sleeping for the
commit time to see if other operations will join
the transaction. The commit time is capped by
the max_batch_time, which defaults to 15000us
(15ms). This optimization can be turned off
entirely by setting max_batch_time to 0.
min_batch_time=usec This parameter sets the commit time (as
described above) to be at least min_batch_time.
It defaults to zero microseconds. Increasing
this parameter may improve the throughput of
multi-threaded, synchronous workloads on very
fast disks, at the cost of increasing latency.
journal_ioprio=prio The I/O priority (from 0 to 7, where 0 is the
highest priorty) which should be used for I/O
operations submitted by kjournald2 during a
commit operation. This defaults to 3, which is
a slightly higher priority than the default I/O
priority.
Data Mode
=========
There are 3 different data modes:
......
......@@ -76,13 +76,13 @@ the fdtable structure -
5. Handling of the file structures is special. Since the look-up
of the fd (fget()/fget_light()) are lock-free, it is possible
that look-up may race with the last put() operation on the
file structure. This is avoided using atomic_inc_not_zero()
file structure. This is avoided using atomic_long_inc_not_zero()
on ->f_count :
rcu_read_lock();
file = fcheck_files(files, fd);
if (file) {
if (atomic_inc_not_zero(&file->f_count))
if (atomic_long_inc_not_zero(&file->f_count))
*fput_needed = 1;
else
/* Didn't get the reference, someone's freed */
......@@ -92,7 +92,7 @@ the fdtable structure -
....
return file;
atomic_inc_not_zero() detects if refcounts is already zero or
atomic_long_inc_not_zero() detects if refcounts is already zero or
goes to zero during increment. If it does, we fail
fget()/fget_light().
......
......@@ -31,7 +31,6 @@ Features which OCFS2 does not support yet:
- quotas
- Directory change notification (F_NOTIFY)
- Distributed Caching (F_SETLEASE/F_GETLEASE/break_lease)
- POSIX ACLs
Mount options
=============
......@@ -79,3 +78,5 @@ inode64 Indicates that Ocfs2 is allowed to create inodes at
bits of significance.
user_xattr (*) Enables Extended User Attributes.
nouser_xattr Disables Extended User Attributes.
acl Enables POSIX Access Control Lists support.
noacl (*) Disables POSIX Access Control Lists support.
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册