提交 f7464060 编写于 作者: L Linus Torvalds

Merge git://github.com/Paragon-Software-Group/linux-ntfs3

Merge NTFSv3 filesystem from Konstantin Komarov:
 "This patch adds NTFS Read-Write driver to fs/ntfs3.

  Having decades of expertise in commercial file systems development and
  huge test coverage, we at Paragon Software GmbH want to make our
  contribution to the Open Source Community by providing implementation
  of NTFS Read-Write driver for the Linux Kernel.

  This is fully functional NTFS Read-Write driver. Current version works
  with NTFS (including v3.1) and normal/compressed/sparse files and
  supports journal replaying.

  We plan to support this version after the codebase once merged, and
  add new features and fix bugs. For example, full journaling support
  over JBD will be added in later updates"

Link: https://lore.kernel.org/lkml/20210729134943.778917-1-almaz.alexandrovich@paragon-software.com/
Link: https://lore.kernel.org/lkml/aa4aa155-b9b2-9099-b7a2-349d8d9d8fbd@paragon-software.com/

* git://github.com/Paragon-Software-Group/linux-ntfs3: (35 commits)
  fs/ntfs3: Change how module init/info messages are displayed
  fs/ntfs3: Remove GPL boilerplates from decompress lib files
  fs/ntfs3: Remove unnecessary condition checking from ntfs_file_read_iter
  fs/ntfs3: Fix integer overflow in ni_fiemap with fiemap_prep()
  fs/ntfs3: Restyle comments to better align with kernel-doc
  fs/ntfs3: Rework file operations
  fs/ntfs3: Remove fat ioctl's from ntfs3 driver for now
  fs/ntfs3: Restyle comments to better align with kernel-doc
  fs/ntfs3: Fix error handling in indx_insert_into_root()
  fs/ntfs3: Potential NULL dereference in hdr_find_split()
  fs/ntfs3: Fix error code in indx_add_allocate()
  fs/ntfs3: fix an error code in ntfs_get_acl_ex()
  fs/ntfs3: add checks for allocation failure
  fs/ntfs3: Use kcalloc/kmalloc_array over kzalloc/kmalloc
  fs/ntfs3: Do not use driver own alloc wrappers
  fs/ntfs3: Use kernel ALIGN macros over driver specific
  fs/ntfs3: Restyle comment block in ni_parse_reparse()
  fs/ntfs3: Remove unused including <linux/version.h>
  fs/ntfs3: Fix fall-through warnings for Clang
  fs/ntfs3: Fix one none utf8 char in source file
  ...
......@@ -101,6 +101,7 @@ Documentation for filesystem implementations.
nilfs2
nfs/index
ntfs
ntfs3
ocfs2
ocfs2-online-filecheck
omfs
......
.. SPDX-License-Identifier: GPL-2.0
=====
NTFS3
=====
Summary and Features
====================
NTFS3 is fully functional NTFS Read-Write driver. The driver works with
NTFS versions up to 3.1, normal/compressed/sparse files
and journal replaying. File system type to use on mount is 'ntfs3'.
- This driver implements NTFS read/write support for normal, sparse and
compressed files.
- Supports native journal replaying;
- Supports extended attributes
Predefined extended attributes:
- 'system.ntfs_security' gets/sets security
descriptor (SECURITY_DESCRIPTOR_RELATIVE)
- 'system.ntfs_attrib' gets/sets ntfs file/dir attributes.
Note: applied to empty files, this allows to switch type between
sparse(0x200), compressed(0x800) and normal;
- Supports NFS export of mounted NTFS volumes.
Mount Options
=============
The list below describes mount options supported by NTFS3 driver in addition to
generic ones.
===============================================================================
nls=name This option informs the driver how to interpret path
strings and translate them to Unicode and back. If
this option is not set, the default codepage will be
used (CONFIG_NLS_DEFAULT).
Examples:
'nls=utf8'
uid=
gid=
umask= Controls the default permissions for files/directories created
after the NTFS volume is mounted.
fmask=
dmask= Instead of specifying umask which applies both to
files and directories, fmask applies only to files and
dmask only to directories.
nohidden Files with the Windows-specific HIDDEN (FILE_ATTRIBUTE_HIDDEN)
attribute will not be shown under Linux.
sys_immutable Files with the Windows-specific SYSTEM
(FILE_ATTRIBUTE_SYSTEM) attribute will be marked as system
immutable files.
discard Enable support of the TRIM command for improved performance
on delete operations, which is recommended for use with the
solid-state drives (SSD).
force Forces the driver to mount partitions even if 'dirty' flag
(volume dirty) is set. Not recommended for use.
sparse Create new files as "sparse".
showmeta Use this parameter to show all meta-files (System Files) on
a mounted NTFS partition.
By default, all meta-files are hidden.
prealloc Preallocate space for files excessively when file size is
increasing on writes. Decreases fragmentation in case of
parallel write operations to different files.
no_acs_rules "No access rules" mount option sets access rights for
files/folders to 777 and owner/group to root. This mount
option absorbs all other permissions:
- permissions change for files/folders will be reported
as successful, but they will remain 777;
- owner/group change will be reported as successful, but
they will stay as root
acl Support POSIX ACLs (Access Control Lists). Effective if
supported by Kernel. Not to be confused with NTFS ACLs.
The option specified as acl enables support for POSIX ACLs.
noatime All files and directories will not update their last access
time attribute if a partition is mounted with this parameter.
This option can speed up file system operation.
===============================================================================
ToDo list
=========
- Full journaling support (currently journal replaying is supported) over JBD.
References
==========
https://www.paragon-software.com/home/ntfs-linux-professional/
- Commercial version of the NTFS driver for Linux.
almaz.alexandrovich@paragon-software.com
- Direct e-mail address for feedback and requests on the NTFS3 implementation.
......@@ -13340,6 +13340,15 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/aia21/ntfs.git
F: Documentation/filesystems/ntfs.rst
F: fs/ntfs/
NTFS3 FILESYSTEM
M: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
L: ntfs3@lists.linux.dev
S: Supported
W: http://www.paragon-software.com/
T: git https://github.com/Paragon-Software-Group/linux-ntfs3.git
F: Documentation/filesystems/ntfs3.rst
F: fs/ntfs3/
NUBUS SUBSYSTEM
M: Finn Thain <fthain@linux-m68k.org>
L: linux-m68k@lists.linux-m68k.org
......
......@@ -136,6 +136,7 @@ menu "DOS/FAT/EXFAT/NT Filesystems"
source "fs/fat/Kconfig"
source "fs/exfat/Kconfig"
source "fs/ntfs/Kconfig"
source "fs/ntfs3/Kconfig"
endmenu
endif # BLOCK
......
......@@ -101,6 +101,7 @@ obj-$(CONFIG_CIFS) += cifs/
obj-$(CONFIG_SMB_SERVER) += ksmbd/
obj-$(CONFIG_HPFS_FS) += hpfs/
obj-$(CONFIG_NTFS_FS) += ntfs/
obj-$(CONFIG_NTFS3_FS) += ntfs3/
obj-$(CONFIG_UFS_FS) += ufs/
obj-$(CONFIG_EFS_FS) += efs/
obj-$(CONFIG_JFFS2_FS) += jffs2/
......
# SPDX-License-Identifier: GPL-2.0-only
config NTFS3_FS
tristate "NTFS Read-Write file system support"
select NLS
help
Windows OS native file system (NTFS) support up to NTFS version 3.1.
Y or M enables the NTFS3 driver with full features enabled (read,
write, journal replaying, sparse/compressed files support).
File system type to use on mount is "ntfs3". Module name (M option)
is also "ntfs3".
Documentation: <file:Documentation/filesystems/ntfs3.rst>
config NTFS3_64BIT_CLUSTER
bool "64 bits per NTFS clusters"
depends on NTFS3_FS && 64BIT
help
Windows implementation of ntfs.sys uses 32 bits per clusters.
If activated 64 bits per clusters you will be able to use 4k cluster
for 16T+ volumes. Windows will not be able to mount such volumes.
It is recommended to say N here.
config NTFS3_LZX_XPRESS
bool "activate support of external compressions lzx/xpress"
depends on NTFS3_FS
help
In Windows 10 one can use command "compact" to compress any files.
4 possible variants of compression are: xpress4k, xpress8k, xpress16k and lzx.
If activated you will be able to read such files correctly.
It is recommended to say Y here.
config NTFS3_FS_POSIX_ACL
bool "NTFS POSIX Access Control Lists"
depends on NTFS3_FS
select FS_POSIX_ACL
help
POSIX Access Control Lists (ACLs) support additional access rights
for users and groups beyond the standard owner/group/world scheme,
and this option selects support for ACLs specifically for ntfs
filesystems.
NOTE: this is linux only feature. Windows will ignore these ACLs.
If you don't know what Access Control Lists are, say N.
# SPDX-License-Identifier: GPL-2.0
#
# Makefile for the ntfs3 filesystem support.
#
# to check robot warnings
ccflags-y += -Wint-to-pointer-cast \
$(call cc-option,-Wunused-but-set-variable,-Wunused-const-variable) \
$(call cc-option,-Wold-style-declaration,-Wout-of-line-declaration)
obj-$(CONFIG_NTFS3_FS) += ntfs3.o
ntfs3-y := attrib.o \
attrlist.o \
bitfunc.o \
bitmap.o \
dir.o \
fsntfs.o \
frecord.o \
file.o \
fslog.o \
inode.o \
index.o \
lznt.o \
namei.o \
record.o \
run.o \
super.o \
upcase.o \
xattr.o
ntfs3-$(CONFIG_NTFS3_LZX_XPRESS) += $(addprefix lib/,\
decompress_common.o \
lzx_decompress.o \
xpress_decompress.o \
)
\ No newline at end of file
此差异已折叠。
// SPDX-License-Identifier: GPL-2.0
/*
*
* Copyright (C) 2019-2021 Paragon Software GmbH, All rights reserved.
*
*/
#include <linux/blkdev.h>
#include <linux/buffer_head.h>
#include <linux/fs.h>
#include <linux/nls.h>
#include "debug.h"
#include "ntfs.h"
#include "ntfs_fs.h"
/*
* al_is_valid_le
*
* Return: True if @le is valid.
*/
static inline bool al_is_valid_le(const struct ntfs_inode *ni,
struct ATTR_LIST_ENTRY *le)
{
if (!le || !ni->attr_list.le || !ni->attr_list.size)
return false;
return PtrOffset(ni->attr_list.le, le) + le16_to_cpu(le->size) <=
ni->attr_list.size;
}
void al_destroy(struct ntfs_inode *ni)
{
run_close(&ni->attr_list.run);
kfree(ni->attr_list.le);
ni->attr_list.le = NULL;
ni->attr_list.size = 0;
ni->attr_list.dirty = false;
}
/*
* ntfs_load_attr_list
*
* This method makes sure that the ATTRIB list, if present,
* has been properly set up.
*/
int ntfs_load_attr_list(struct ntfs_inode *ni, struct ATTRIB *attr)
{
int err;
size_t lsize;
void *le = NULL;
if (ni->attr_list.size)
return 0;
if (!attr->non_res) {
lsize = le32_to_cpu(attr->res.data_size);
le = kmalloc(al_aligned(lsize), GFP_NOFS);
if (!le) {
err = -ENOMEM;
goto out;
}
memcpy(le, resident_data(attr), lsize);
} else if (attr->nres.svcn) {
err = -EINVAL;
goto out;
} else {
u16 run_off = le16_to_cpu(attr->nres.run_off);
lsize = le64_to_cpu(attr->nres.data_size);
run_init(&ni->attr_list.run);
err = run_unpack_ex(&ni->attr_list.run, ni->mi.sbi, ni->mi.rno,
0, le64_to_cpu(attr->nres.evcn), 0,
Add2Ptr(attr, run_off),
le32_to_cpu(attr->size) - run_off);
if (err < 0)
goto out;
le = kmalloc(al_aligned(lsize), GFP_NOFS);
if (!le) {
err = -ENOMEM;
goto out;
}
err = ntfs_read_run_nb(ni->mi.sbi, &ni->attr_list.run, 0, le,
lsize, NULL);
if (err)
goto out;
}
ni->attr_list.size = lsize;
ni->attr_list.le = le;
return 0;
out:
ni->attr_list.le = le;
al_destroy(ni);
return err;
}
/*
* al_enumerate
*
* Return:
* * The next list le.
* * If @le is NULL then return the first le.
*/
struct ATTR_LIST_ENTRY *al_enumerate(struct ntfs_inode *ni,
struct ATTR_LIST_ENTRY *le)
{
size_t off;
u16 sz;
if (!le) {
le = ni->attr_list.le;
} else {
sz = le16_to_cpu(le->size);
if (sz < sizeof(struct ATTR_LIST_ENTRY)) {
/* Impossible 'cause we should not return such le. */
return NULL;
}
le = Add2Ptr(le, sz);
}
/* Check boundary. */
off = PtrOffset(ni->attr_list.le, le);
if (off + sizeof(struct ATTR_LIST_ENTRY) > ni->attr_list.size) {
/* The regular end of list. */
return NULL;
}
sz = le16_to_cpu(le->size);
/* Check le for errors. */
if (sz < sizeof(struct ATTR_LIST_ENTRY) ||
off + sz > ni->attr_list.size ||
sz < le->name_off + le->name_len * sizeof(short)) {
return NULL;
}
return le;
}
/*
* al_find_le
*
* Find the first le in the list which matches type, name and VCN.
*
* Return: NULL if not found.
*/
struct ATTR_LIST_ENTRY *al_find_le(struct ntfs_inode *ni,
struct ATTR_LIST_ENTRY *le,
const struct ATTRIB *attr)
{
CLST svcn = attr_svcn(attr);
return al_find_ex(ni, le, attr->type, attr_name(attr), attr->name_len,
&svcn);
}
/*
* al_find_ex
*
* Find the first le in the list which matches type, name and VCN.
*
* Return: NULL if not found.
*/
struct ATTR_LIST_ENTRY *al_find_ex(struct ntfs_inode *ni,
struct ATTR_LIST_ENTRY *le,
enum ATTR_TYPE type, const __le16 *name,
u8 name_len, const CLST *vcn)
{
struct ATTR_LIST_ENTRY *ret = NULL;
u32 type_in = le32_to_cpu(type);
while ((le = al_enumerate(ni, le))) {
u64 le_vcn;
int diff = le32_to_cpu(le->type) - type_in;
/* List entries are sorted by type, name and VCN. */
if (diff < 0)
continue;
if (diff > 0)
return ret;
if (le->name_len != name_len)
continue;
le_vcn = le64_to_cpu(le->vcn);
if (!le_vcn) {
/*
* Compare entry names only for entry with vcn == 0.
*/
diff = ntfs_cmp_names(le_name(le), name_len, name,
name_len, ni->mi.sbi->upcase,
true);
if (diff < 0)
continue;
if (diff > 0)
return ret;
}
if (!vcn)
return le;
if (*vcn == le_vcn)
return le;
if (*vcn < le_vcn)
return ret;
ret = le;
}
return ret;
}
/*
* al_find_le_to_insert
*
* Find the first list entry which matches type, name and VCN.
*/
static struct ATTR_LIST_ENTRY *al_find_le_to_insert(struct ntfs_inode *ni,
enum ATTR_TYPE type,
const __le16 *name,
u8 name_len, CLST vcn)
{
struct ATTR_LIST_ENTRY *le = NULL, *prev;
u32 type_in = le32_to_cpu(type);
/* List entries are sorted by type, name and VCN. */
while ((le = al_enumerate(ni, prev = le))) {
int diff = le32_to_cpu(le->type) - type_in;
if (diff < 0)
continue;
if (diff > 0)
return le;
if (!le->vcn) {
/*
* Compare entry names only for entry with vcn == 0.
*/
diff = ntfs_cmp_names(le_name(le), le->name_len, name,
name_len, ni->mi.sbi->upcase,
true);
if (diff < 0)
continue;
if (diff > 0)
return le;
}
if (le64_to_cpu(le->vcn) >= vcn)
return le;
}
return prev ? Add2Ptr(prev, le16_to_cpu(prev->size)) : ni->attr_list.le;
}
/*
* al_add_le
*
* Add an "attribute list entry" to the list.
*/
int al_add_le(struct ntfs_inode *ni, enum ATTR_TYPE type, const __le16 *name,
u8 name_len, CLST svcn, __le16 id, const struct MFT_REF *ref,
struct ATTR_LIST_ENTRY **new_le)
{
int err;
struct ATTRIB *attr;
struct ATTR_LIST_ENTRY *le;
size_t off;
u16 sz;
size_t asize, new_asize, old_size;
u64 new_size;
typeof(ni->attr_list) *al = &ni->attr_list;
/*
* Compute the size of the new 'le'
*/
sz = le_size(name_len);
old_size = al->size;
new_size = old_size + sz;
asize = al_aligned(old_size);
new_asize = al_aligned(new_size);
/* Scan forward to the point at which the new 'le' should be inserted. */
le = al_find_le_to_insert(ni, type, name, name_len, svcn);
off = PtrOffset(al->le, le);
if (new_size > asize) {
void *ptr = kmalloc(new_asize, GFP_NOFS);
if (!ptr)
return -ENOMEM;
memcpy(ptr, al->le, off);
memcpy(Add2Ptr(ptr, off + sz), le, old_size - off);
le = Add2Ptr(ptr, off);
kfree(al->le);
al->le = ptr;
} else {
memmove(Add2Ptr(le, sz), le, old_size - off);
}
*new_le = le;
al->size = new_size;
le->type = type;
le->size = cpu_to_le16(sz);
le->name_len = name_len;
le->name_off = offsetof(struct ATTR_LIST_ENTRY, name);
le->vcn = cpu_to_le64(svcn);
le->ref = *ref;
le->id = id;
memcpy(le->name, name, sizeof(short) * name_len);
err = attr_set_size(ni, ATTR_LIST, NULL, 0, &al->run, new_size,
&new_size, true, &attr);
if (err) {
/* Undo memmove above. */
memmove(le, Add2Ptr(le, sz), old_size - off);
al->size = old_size;
return err;
}
al->dirty = true;
if (attr && attr->non_res) {
err = ntfs_sb_write_run(ni->mi.sbi, &al->run, 0, al->le,
al->size);
if (err)
return err;
al->dirty = false;
}
return 0;
}
/*
* al_remove_le - Remove @le from attribute list.
*/
bool al_remove_le(struct ntfs_inode *ni, struct ATTR_LIST_ENTRY *le)
{
u16 size;
size_t off;
typeof(ni->attr_list) *al = &ni->attr_list;
if (!al_is_valid_le(ni, le))
return false;
/* Save on stack the size of 'le' */
size = le16_to_cpu(le->size);
off = PtrOffset(al->le, le);
memmove(le, Add2Ptr(le, size), al->size - (off + size));
al->size -= size;
al->dirty = true;
return true;
}
/*
* al_delete_le - Delete first le from the list which matches its parameters.
*/
bool al_delete_le(struct ntfs_inode *ni, enum ATTR_TYPE type, CLST vcn,
const __le16 *name, size_t name_len,
const struct MFT_REF *ref)
{
u16 size;
struct ATTR_LIST_ENTRY *le;
size_t off;
typeof(ni->attr_list) *al = &ni->attr_list;
/* Scan forward to the first le that matches the input. */
le = al_find_ex(ni, NULL, type, name, name_len, &vcn);
if (!le)
return false;
off = PtrOffset(al->le, le);
next:
if (off >= al->size)
return false;
if (le->type != type)
return false;
if (le->name_len != name_len)
return false;
if (name_len && ntfs_cmp_names(le_name(le), name_len, name, name_len,
ni->mi.sbi->upcase, true))
return false;
if (le64_to_cpu(le->vcn) != vcn)
return false;
/*
* The caller specified a segment reference, so we have to
* scan through the matching entries until we find that segment
* reference or we run of matching entries.
*/
if (ref && memcmp(ref, &le->ref, sizeof(*ref))) {
off += le16_to_cpu(le->size);
le = Add2Ptr(al->le, off);
goto next;
}
/* Save on stack the size of 'le'. */
size = le16_to_cpu(le->size);
/* Delete the le. */
memmove(le, Add2Ptr(le, size), al->size - (off + size));
al->size -= size;
al->dirty = true;
return true;
}
int al_update(struct ntfs_inode *ni)
{
int err;
struct ATTRIB *attr;
typeof(ni->attr_list) *al = &ni->attr_list;
if (!al->dirty || !al->size)
return 0;
/*
* Attribute list increased on demand in al_add_le.
* Attribute list decreased here.
*/
err = attr_set_size(ni, ATTR_LIST, NULL, 0, &al->run, al->size, NULL,
false, &attr);
if (err)
goto out;
if (!attr->non_res) {
memcpy(resident_data(attr), al->le, al->size);
} else {
err = ntfs_sb_write_run(ni->mi.sbi, &al->run, 0, al->le,
al->size);
if (err)
goto out;
attr->nres.valid_size = attr->nres.data_size;
}
ni->mi.dirty = true;
al->dirty = false;
out:
return err;
}
// SPDX-License-Identifier: GPL-2.0
/*
*
* Copyright (C) 2019-2021 Paragon Software GmbH, All rights reserved.
*
*/
#include <linux/blkdev.h>
#include <linux/buffer_head.h>
#include <linux/fs.h>
#include <linux/nls.h>
#include "debug.h"
#include "ntfs.h"
#include "ntfs_fs.h"
#define BITS_IN_SIZE_T (sizeof(size_t) * 8)
/*
* fill_mask[i] - first i bits are '1' , i = 0,1,2,3,4,5,6,7,8
* fill_mask[i] = 0xFF >> (8-i)
*/
static const u8 fill_mask[] = { 0x00, 0x01, 0x03, 0x07, 0x0F,
0x1F, 0x3F, 0x7F, 0xFF };
/*
* zero_mask[i] - first i bits are '0' , i = 0,1,2,3,4,5,6,7,8
* zero_mask[i] = 0xFF << i
*/
static const u8 zero_mask[] = { 0xFF, 0xFE, 0xFC, 0xF8, 0xF0,
0xE0, 0xC0, 0x80, 0x00 };
/*
* are_bits_clear
*
* Return: True if all bits [bit, bit+nbits) are zeros "0".
*/
bool are_bits_clear(const ulong *lmap, size_t bit, size_t nbits)
{
size_t pos = bit & 7;
const u8 *map = (u8 *)lmap + (bit >> 3);
if (pos) {
if (8 - pos >= nbits)
return !nbits || !(*map & fill_mask[pos + nbits] &
zero_mask[pos]);
if (*map++ & zero_mask[pos])
return false;
nbits -= 8 - pos;
}
pos = ((size_t)map) & (sizeof(size_t) - 1);
if (pos) {
pos = sizeof(size_t) - pos;
if (nbits >= pos * 8) {
for (nbits -= pos * 8; pos; pos--, map++) {
if (*map)
return false;
}
}
}
for (pos = nbits / BITS_IN_SIZE_T; pos; pos--, map += sizeof(size_t)) {
if (*((size_t *)map))
return false;
}
for (pos = (nbits % BITS_IN_SIZE_T) >> 3; pos; pos--, map++) {
if (*map)
return false;
}
pos = nbits & 7;
if (pos && (*map & fill_mask[pos]))
return false;
return true;
}
/*
* are_bits_set
*
* Return: True if all bits [bit, bit+nbits) are ones "1".
*/
bool are_bits_set(const ulong *lmap, size_t bit, size_t nbits)
{
u8 mask;
size_t pos = bit & 7;
const u8 *map = (u8 *)lmap + (bit >> 3);
if (pos) {
if (8 - pos >= nbits) {
mask = fill_mask[pos + nbits] & zero_mask[pos];
return !nbits || (*map & mask) == mask;
}
mask = zero_mask[pos];
if ((*map++ & mask) != mask)
return false;
nbits -= 8 - pos;
}
pos = ((size_t)map) & (sizeof(size_t) - 1);
if (pos) {
pos = sizeof(size_t) - pos;
if (nbits >= pos * 8) {
for (nbits -= pos * 8; pos; pos--, map++) {
if (*map != 0xFF)
return false;
}
}
}
for (pos = nbits / BITS_IN_SIZE_T; pos; pos--, map += sizeof(size_t)) {
if (*((size_t *)map) != MINUS_ONE_T)
return false;
}
for (pos = (nbits % BITS_IN_SIZE_T) >> 3; pos; pos--, map++) {
if (*map != 0xFF)
return false;
}
pos = nbits & 7;
if (pos) {
u8 mask = fill_mask[pos];
if ((*map & mask) != mask)
return false;
}
return true;
}
此差异已折叠。
/* SPDX-License-Identifier: GPL-2.0 */
/*
*
* Copyright (C) 2019-2021 Paragon Software GmbH, All rights reserved.
*
* Useful functions for debugging.
*
*/
// clang-format off
#ifndef _LINUX_NTFS3_DEBUG_H
#define _LINUX_NTFS3_DEBUG_H
#ifndef Add2Ptr
#define Add2Ptr(P, I) ((void *)((u8 *)(P) + (I)))
#define PtrOffset(B, O) ((size_t)((size_t)(O) - (size_t)(B)))
#endif
#ifdef CONFIG_PRINTK
__printf(2, 3)
void ntfs_printk(const struct super_block *sb, const char *fmt, ...);
__printf(2, 3)
void ntfs_inode_printk(struct inode *inode, const char *fmt, ...);
#else
static inline __printf(2, 3)
void ntfs_printk(const struct super_block *sb, const char *fmt, ...)
{
}
static inline __printf(2, 3)
void ntfs_inode_printk(struct inode *inode, const char *fmt, ...)
{
}
#endif
/*
* Logging macros. Thanks Joe Perches <joe@perches.com> for implementation.
*/
#define ntfs_err(sb, fmt, ...) ntfs_printk(sb, KERN_ERR fmt, ##__VA_ARGS__)
#define ntfs_warn(sb, fmt, ...) ntfs_printk(sb, KERN_WARNING fmt, ##__VA_ARGS__)
#define ntfs_info(sb, fmt, ...) ntfs_printk(sb, KERN_INFO fmt, ##__VA_ARGS__)
#define ntfs_notice(sb, fmt, ...) \
ntfs_printk(sb, KERN_NOTICE fmt, ##__VA_ARGS__)
#define ntfs_inode_err(inode, fmt, ...) \
ntfs_inode_printk(inode, KERN_ERR fmt, ##__VA_ARGS__)
#define ntfs_inode_warn(inode, fmt, ...) \
ntfs_inode_printk(inode, KERN_WARNING fmt, ##__VA_ARGS__)
#endif /* _LINUX_NTFS3_DEBUG_H */
// clang-format on
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* decompress_common.c - Code shared by the XPRESS and LZX decompressors
*
* Copyright (C) 2015 Eric Biggers
*/
#include "decompress_common.h"
/*
* make_huffman_decode_table() -
*
* Build a decoding table for a canonical prefix code, or "Huffman code".
*
* This is an internal function, not part of the library API!
*
* This takes as input the length of the codeword for each symbol in the
* alphabet and produces as output a table that can be used for fast
* decoding of prefix-encoded symbols using read_huffsym().
*
* Strictly speaking, a canonical prefix code might not be a Huffman
* code. But this algorithm will work either way; and in fact, since
* Huffman codes are defined in terms of symbol frequencies, there is no
* way for the decompressor to know whether the code is a true Huffman
* code or not until all symbols have been decoded.
*
* Because the prefix code is assumed to be "canonical", it can be
* reconstructed directly from the codeword lengths. A prefix code is
* canonical if and only if a longer codeword never lexicographically
* precedes a shorter codeword, and the lexicographic ordering of
* codewords of the same length is the same as the lexicographic ordering
* of the corresponding symbols. Consequently, we can sort the symbols
* primarily by codeword length and secondarily by symbol value, then
* reconstruct the prefix code by generating codewords lexicographically
* in that order.
*
* This function does not, however, generate the prefix code explicitly.
* Instead, it directly builds a table for decoding symbols using the
* code. The basic idea is this: given the next 'max_codeword_len' bits
* in the input, we can look up the decoded symbol by indexing a table
* containing 2**max_codeword_len entries. A codeword with length
* 'max_codeword_len' will have exactly one entry in this table, whereas
* a codeword shorter than 'max_codeword_len' will have multiple entries
* in this table. Precisely, a codeword of length n will be represented
* by 2**(max_codeword_len - n) entries in this table. The 0-based index
* of each such entry will contain the corresponding codeword as a prefix
* when zero-padded on the left to 'max_codeword_len' binary digits.
*
* That's the basic idea, but we implement two optimizations regarding
* the format of the decode table itself:
*
* - For many compression formats, the maximum codeword length is too
* long for it to be efficient to build the full decoding table
* whenever a new prefix code is used. Instead, we can build the table
* using only 2**table_bits entries, where 'table_bits' is some number
* less than or equal to 'max_codeword_len'. Then, only codewords of
* length 'table_bits' and shorter can be directly looked up. For
* longer codewords, the direct lookup instead produces the root of a
* binary tree. Using this tree, the decoder can do traditional
* bit-by-bit decoding of the remainder of the codeword. Child nodes
* are allocated in extra entries at the end of the table; leaf nodes
* contain symbols. Note that the long-codeword case is, in general,
* not performance critical, since in Huffman codes the most frequently
* used symbols are assigned the shortest codeword lengths.
*
* - When we decode a symbol using a direct lookup of the table, we still
* need to know its length so that the bitstream can be advanced by the
* appropriate number of bits. The simple solution is to simply retain
* the 'lens' array and use the decoded symbol as an index into it.
* However, this requires two separate array accesses in the fast path.
* The optimization is to store the length directly in the decode
* table. We use the bottom 11 bits for the symbol and the top 5 bits
* for the length. In addition, to combine this optimization with the
* previous one, we introduce a special case where the top 2 bits of
* the length are both set if the entry is actually the root of a
* binary tree.
*
* @decode_table:
* The array in which to create the decoding table. This must have
* a length of at least ((2**table_bits) + 2 * num_syms) entries.
*
* @num_syms:
* The number of symbols in the alphabet; also, the length of the
* 'lens' array. Must be less than or equal to 2048.
*
* @table_bits:
* The order of the decode table size, as explained above. Must be
* less than or equal to 13.
*
* @lens:
* An array of length @num_syms, indexable by symbol, that gives the
* length of the codeword, in bits, for that symbol. The length can
* be 0, which means that the symbol does not have a codeword
* assigned.
*
* @max_codeword_len:
* The longest codeword length allowed in the compression format.
* All entries in 'lens' must be less than or equal to this value.
* This must be less than or equal to 23.
*
* @working_space
* A temporary array of length '2 * (max_codeword_len + 1) +
* num_syms'.
*
* Returns 0 on success, or -1 if the lengths do not form a valid prefix
* code.
*/
int make_huffman_decode_table(u16 decode_table[], const u32 num_syms,
const u32 table_bits, const u8 lens[],
const u32 max_codeword_len,
u16 working_space[])
{
const u32 table_num_entries = 1 << table_bits;
u16 * const len_counts = &working_space[0];
u16 * const offsets = &working_space[1 * (max_codeword_len + 1)];
u16 * const sorted_syms = &working_space[2 * (max_codeword_len + 1)];
int left;
void *decode_table_ptr;
u32 sym_idx;
u32 codeword_len;
u32 stores_per_loop;
u32 decode_table_pos;
u32 len;
u32 sym;
/* Count how many symbols have each possible codeword length.
* Note that a length of 0 indicates the corresponding symbol is not
* used in the code and therefore does not have a codeword.
*/
for (len = 0; len <= max_codeword_len; len++)
len_counts[len] = 0;
for (sym = 0; sym < num_syms; sym++)
len_counts[lens[sym]]++;
/* We can assume all lengths are <= max_codeword_len, but we
* cannot assume they form a valid prefix code. A codeword of
* length n should require a proportion of the codespace equaling
* (1/2)^n. The code is valid if and only if the codespace is
* exactly filled by the lengths, by this measure.
*/
left = 1;
for (len = 1; len <= max_codeword_len; len++) {
left <<= 1;
left -= len_counts[len];
if (left < 0) {
/* The lengths overflow the codespace; that is, the code
* is over-subscribed.
*/
return -1;
}
}
if (left) {
/* The lengths do not fill the codespace; that is, they form an
* incomplete set.
*/
if (left == (1 << max_codeword_len)) {
/* The code is completely empty. This is arguably
* invalid, but in fact it is valid in LZX and XPRESS,
* so we must allow it. By definition, no symbols can
* be decoded with an empty code. Consequently, we
* technically don't even need to fill in the decode
* table. However, to avoid accessing uninitialized
* memory if the algorithm nevertheless attempts to
* decode symbols using such a code, we zero out the
* decode table.
*/
memset(decode_table, 0,
table_num_entries * sizeof(decode_table[0]));
return 0;
}
return -1;
}
/* Sort the symbols primarily by length and secondarily by symbol order.
*/
/* Initialize 'offsets' so that offsets[len] for 1 <= len <=
* max_codeword_len is the number of codewords shorter than 'len' bits.
*/
offsets[1] = 0;
for (len = 1; len < max_codeword_len; len++)
offsets[len + 1] = offsets[len] + len_counts[len];
/* Use the 'offsets' array to sort the symbols. Note that we do not
* include symbols that are not used in the code. Consequently, fewer
* than 'num_syms' entries in 'sorted_syms' may be filled.
*/
for (sym = 0; sym < num_syms; sym++)
if (lens[sym])
sorted_syms[offsets[lens[sym]]++] = sym;
/* Fill entries for codewords with length <= table_bits
* --- that is, those short enough for a direct mapping.
*
* The table will start with entries for the shortest codeword(s), which
* have the most entries. From there, the number of entries per
* codeword will decrease.
*/
decode_table_ptr = decode_table;
sym_idx = 0;
codeword_len = 1;
stores_per_loop = (1 << (table_bits - codeword_len));
for (; stores_per_loop != 0; codeword_len++, stores_per_loop >>= 1) {
u32 end_sym_idx = sym_idx + len_counts[codeword_len];
for (; sym_idx < end_sym_idx; sym_idx++) {
u16 entry;
u16 *p;
u32 n;
entry = ((u32)codeword_len << 11) | sorted_syms[sym_idx];
p = (u16 *)decode_table_ptr;
n = stores_per_loop;
do {
*p++ = entry;
} while (--n);
decode_table_ptr = p;
}
}
/* If we've filled in the entire table, we are done. Otherwise,
* there are codewords longer than table_bits for which we must
* generate binary trees.
*/
decode_table_pos = (u16 *)decode_table_ptr - decode_table;
if (decode_table_pos != table_num_entries) {
u32 j;
u32 next_free_tree_slot;
u32 cur_codeword;
/* First, zero out the remaining entries. This is
* necessary so that these entries appear as
* "unallocated" in the next part. Each of these entries
* will eventually be filled with the representation of
* the root node of a binary tree.
*/
j = decode_table_pos;
do {
decode_table[j] = 0;
} while (++j != table_num_entries);
/* We allocate child nodes starting at the end of the
* direct lookup table. Note that there should be
* 2*num_syms extra entries for this purpose, although
* fewer than this may actually be needed.
*/
next_free_tree_slot = table_num_entries;
/* Iterate through each codeword with length greater than
* 'table_bits', primarily in order of codeword length
* and secondarily in order of symbol.
*/
for (cur_codeword = decode_table_pos << 1;
codeword_len <= max_codeword_len;
codeword_len++, cur_codeword <<= 1) {
u32 end_sym_idx = sym_idx + len_counts[codeword_len];
for (; sym_idx < end_sym_idx; sym_idx++, cur_codeword++) {
/* 'sorted_sym' is the symbol represented by the
* codeword.
*/
u32 sorted_sym = sorted_syms[sym_idx];
u32 extra_bits = codeword_len - table_bits;
u32 node_idx = cur_codeword >> extra_bits;
/* Go through each bit of the current codeword
* beyond the prefix of length @table_bits and
* walk the appropriate binary tree, allocating
* any slots that have not yet been allocated.
*
* Note that the 'pointer' entry to the binary
* tree, which is stored in the direct lookup
* portion of the table, is represented
* identically to other internal (non-leaf)
* nodes of the binary tree; it can be thought
* of as simply the root of the tree. The
* representation of these internal nodes is
* simply the index of the left child combined
* with the special bits 0xC000 to distinguish
* the entry from direct mapping and leaf node
* entries.
*/
do {
/* At least one bit remains in the
* codeword, but the current node is an
* unallocated leaf. Change it to an
* internal node.
*/
if (decode_table[node_idx] == 0) {
decode_table[node_idx] =
next_free_tree_slot | 0xC000;
decode_table[next_free_tree_slot++] = 0;
decode_table[next_free_tree_slot++] = 0;
}
/* Go to the left child if the next bit
* in the codeword is 0; otherwise go to
* the right child.
*/
node_idx = decode_table[node_idx] & 0x3FFF;
--extra_bits;
node_idx += (cur_codeword >> extra_bits) & 1;
} while (extra_bits != 0);
/* We've traversed the tree using the entire
* codeword, and we're now at the entry where
* the actual symbol will be stored. This is
* distinguished from internal nodes by not
* having its high two bits set.
*/
decode_table[node_idx] = sorted_sym;
}
}
}
return 0;
}
此差异已折叠。
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Adapted for linux kernel by Alexander Mamaev:
* - remove implementations of get_unaligned_
* - assume GCC is always defined
* - ISO C90
* - linux kernel code style
*/
/* globals from xpress_decompress.c */
struct xpress_decompressor *xpress_allocate_decompressor(void);
void xpress_free_decompressor(struct xpress_decompressor *d);
int xpress_decompress(struct xpress_decompressor *__restrict d,
const void *__restrict compressed_data,
size_t compressed_size,
void *__restrict uncompressed_data,
size_t uncompressed_size);
/* globals from lzx_decompress.c */
struct lzx_decompressor *lzx_allocate_decompressor(void);
void lzx_free_decompressor(struct lzx_decompressor *d);
int lzx_decompress(struct lzx_decompressor *__restrict d,
const void *__restrict compressed_data,
size_t compressed_size, void *__restrict uncompressed_data,
size_t uncompressed_size);
此差异已折叠。
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* xpress_decompress.c - A decompressor for the XPRESS compression format
* (Huffman variant), which can be used in "System Compressed" files. This is
* based on the code from wimlib.
*
* Copyright (C) 2015 Eric Biggers
*/
#include "decompress_common.h"
#include "lib.h"
#define XPRESS_NUM_SYMBOLS 512
#define XPRESS_MAX_CODEWORD_LEN 15
#define XPRESS_MIN_MATCH_LEN 3
/* This value is chosen for fast decompression. */
#define XPRESS_TABLEBITS 12
/* Reusable heap-allocated memory for XPRESS decompression */
struct xpress_decompressor {
/* The Huffman decoding table */
u16 decode_table[(1 << XPRESS_TABLEBITS) + 2 * XPRESS_NUM_SYMBOLS];
/* An array that maps symbols to codeword lengths */
u8 lens[XPRESS_NUM_SYMBOLS];
/* Temporary space for make_huffman_decode_table() */
u16 working_space[2 * (1 + XPRESS_MAX_CODEWORD_LEN) +
XPRESS_NUM_SYMBOLS];
};
/*
* xpress_allocate_decompressor - Allocate an XPRESS decompressor
*
* Return the pointer to the decompressor on success, or return NULL and set
* errno on failure.
*/
struct xpress_decompressor *xpress_allocate_decompressor(void)
{
return kmalloc(sizeof(struct xpress_decompressor), GFP_NOFS);
}
/*
* xpress_decompress - Decompress a buffer of XPRESS-compressed data
*
* @decompressor: A decompressor that was allocated with
* xpress_allocate_decompressor()
* @compressed_data: The buffer of data to decompress
* @compressed_size: Number of bytes of compressed data
* @uncompressed_data: The buffer in which to store the decompressed data
* @uncompressed_size: The number of bytes the data decompresses into
*
* Return 0 on success, or return -1 and set errno on failure.
*/
int xpress_decompress(struct xpress_decompressor *decompressor,
const void *compressed_data, size_t compressed_size,
void *uncompressed_data, size_t uncompressed_size)
{
struct xpress_decompressor *d = decompressor;
const u8 * const in_begin = compressed_data;
u8 * const out_begin = uncompressed_data;
u8 *out_next = out_begin;
u8 * const out_end = out_begin + uncompressed_size;
struct input_bitstream is;
u32 i;
/* Read the Huffman codeword lengths. */
if (compressed_size < XPRESS_NUM_SYMBOLS / 2)
goto invalid;
for (i = 0; i < XPRESS_NUM_SYMBOLS / 2; i++) {
d->lens[i*2 + 0] = in_begin[i] & 0xF;
d->lens[i*2 + 1] = in_begin[i] >> 4;
}
/* Build a decoding table for the Huffman code. */
if (make_huffman_decode_table(d->decode_table, XPRESS_NUM_SYMBOLS,
XPRESS_TABLEBITS, d->lens,
XPRESS_MAX_CODEWORD_LEN,
d->working_space))
goto invalid;
/* Decode the matches and literals. */
init_input_bitstream(&is, in_begin + XPRESS_NUM_SYMBOLS / 2,
compressed_size - XPRESS_NUM_SYMBOLS / 2);
while (out_next != out_end) {
u32 sym;
u32 log2_offset;
u32 length;
u32 offset;
sym = read_huffsym(&is, d->decode_table,
XPRESS_TABLEBITS, XPRESS_MAX_CODEWORD_LEN);
if (sym < 256) {
/* Literal */
*out_next++ = sym;
} else {
/* Match */
length = sym & 0xf;
log2_offset = (sym >> 4) & 0xf;
bitstream_ensure_bits(&is, 16);
offset = ((u32)1 << log2_offset) |
bitstream_pop_bits(&is, log2_offset);
if (length == 0xf) {
length += bitstream_read_byte(&is);
if (length == 0xf + 0xff)
length = bitstream_read_u16(&is);
}
length += XPRESS_MIN_MATCH_LEN;
if (offset > (size_t)(out_next - out_begin))
goto invalid;
if (length > (size_t)(out_end - out_next))
goto invalid;
out_next = lz_copy(out_next, length, offset, out_end,
XPRESS_MIN_MATCH_LEN);
}
}
return 0;
invalid:
return -1;
}
/*
* xpress_free_decompressor - Free an XPRESS decompressor
*
* @decompressor: A decompressor that was allocated with
* xpress_allocate_decompressor(), or NULL.
*/
void xpress_free_decompressor(struct xpress_decompressor *decompressor)
{
kfree(decompressor);
}
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册