- 23 12月, 2018 3 次提交
-
-
由 Dave Watson 提交于
AAD hash only needs to be calculated once for each scatter/gather operation. Move it to its own macro, and call it from GCM_INIT instead of INITIAL_BLOCKS. Signed-off-by: NDave Watson <davejwatson@fb.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
由 Dave Watson 提交于
Add support for 192/256-bit keys using the avx gcm/aes routines. The sse routines were previously updated in e31ac32d (Add support for 192 & 256 bit keys to AESNI RFC4106). Instead of adding an additional loop in the hotpath as in e31ac32d, this diff instead generates separate versions of the code using macros, and the entry routines choose which version once. This results in a 5% performance improvement vs. adding a loop to the hot path. This is the same strategy chosen by the intel isa-l_crypto library. The key size checks are removed from the c code where appropriate. Note that this diff depends on using gcm_context_data - 256 bit keys require 16 HashKeys + 15 expanded keys, which is larger than struct crypto_aes_ctx, so they are stored in struct gcm_context_data. Signed-off-by: NDave Watson <davejwatson@fb.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
由 Dave Watson 提交于
Add the gcm_context_data structure to the avx asm routines. This will be necessary to support both 256 bit keys and scatter/gather. The pre-computed HashKeys are now stored in the gcm_context_data struct, which is expanded to hold the greater number of hashkeys necessary for avx. Loads and stores to the new struct are always done unlaligned to avoid compiler issues, see e5b954e8 "Use unaligned loads from gcm_context_data" Signed-off-by: NDave Watson <davejwatson@fb.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 08 10月, 2018 1 次提交
-
-
由 Eric Biggers 提交于
aesni-intel_glue.c still calls crypto_fpu_init() and crypto_fpu_exit() to register/unregister the "fpu" template. But these functions don't exist anymore, causing a build error. Remove the calls to them. Fixes: 944585a6 ("crypto: x86/aes-ni - remove special handling of AES in PCBC mode") Signed-off-by: NEric Biggers <ebiggers@google.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 05 10月, 2018 1 次提交
-
-
由 Ard Biesheuvel 提交于
For historical reasons, the AES-NI based implementation of the PCBC chaining mode uses a special FPU chaining mode wrapper template to amortize the FPU start/stop overhead over multiple blocks. When this FPU wrapper was introduced, it supported widely used chaining modes such as XTS and CTR (as well as LRW), but currently, PCBC is the only remaining user. Since there are no known users of pcbc(aes) in the kernel, let's remove this special driver, and rely on the generic pcbc driver to encapsulate the AES-NI core cipher. Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 14 9月, 2018 1 次提交
-
-
由 Mikulas Patocka 提交于
This patch fixes gcmaes_crypt_by_sg so that it won't use memory allocation if the data doesn't cross a page boundary. Authenticated encryption may be used by dm-crypt. If the encryption or decryption fails, it would result in I/O error and filesystem corruption. The function gcmaes_crypt_by_sg is using GFP_ATOMIC allocation that can fail anytime. This patch fixes the logic so that it won't attempt the failing allocation if the data doesn't cross a page boundary. Signed-off-by: NMikulas Patocka <mpatocka@redhat.com> Cc: stable@vger.kernel.org Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 22 2月, 2018 3 次提交
-
-
由 Dave Watson 提交于
Add gcmaes_crypt_by_sg routine, that will do scatter/gather by sg. Either src or dst may contain multiple buffers, so iterate over both at the same time if they are different. If the input is the same as the output, iterate only over one. Currently both the AAD and TAG must be linear, so copy them out with scatterlist_map_and_copy. If first buffer contains the entire AAD, we can optimize and not copy. Since the AAD can be any size, if copied it must be on the heap. TAG can be on the stack since it is always < 16 bytes. Only the SSE routines are updated so far, so leave the previous gcmaes_en/decrypt routines, and branch to the sg ones if the keysize is inappropriate for avx, or we are SSE only. Signed-off-by: NDave Watson <davejwatson@fb.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
由 Dave Watson 提交于
The asm macros are all set up now, introduce entry points. GCM_INIT and GCM_COMPLETE have arguments supplied, so that the new scatter/gather entry points don't have to take all the arguments, and only the ones they need. Signed-off-by: NDave Watson <davejwatson@fb.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
由 Dave Watson 提交于
Introduce a gcm_context_data struct that will be used to pass context data between scatter/gather update calls. It is passed as the second argument (after crypto keys), other args are renumbered. Signed-off-by: NDave Watson <davejwatson@fb.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 25 1月, 2018 1 次提交
-
-
由 Stephan Mueller 提交于
GCM can be invoked with a zero destination buffer. This is possible if the AAD and the ciphertext have zero lengths and only the tag exists in the source buffer (i.e. a source buffer cannot be zero). In this case, the GCM cipher only performs the authentication and no decryption operation. When the destination buffer has zero length, it is possible that no page is mapped to the SG pointing to the destination. In this case, sg_page(req->dst) is an invalid access. Therefore, page accesses should only be allowed if the req->dst->length is non-zero which is the indicator that a page must exist. This fixes a crash that can be triggered by user space via AF_ALG. CC: <stable@vger.kernel.org> Signed-off-by: NStephan Mueller <smueller@chronox.de> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 22 12月, 2017 2 次提交
-
-
由 Sabrina Dubroca 提交于
When I added generic-gcm-aes I didn't add a wrapper like the one provided for rfc4106(gcm(aes)). We need to add a cryptd wrapper to fall back on in case the FPU is not available, otherwise we might corrupt the FPU state. Fixes: cce2ea8d ("crypto: aesni - add generic gcm(aes)") Cc: <stable@vger.kernel.org> Reported-by: NIlya Lesokhin <ilyal@mellanox.com> Signed-off-by: NSabrina Dubroca <sd@queasysnail.net> Reviewed-by: NStefano Brivio <sbrivio@redhat.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
由 Sabrina Dubroca 提交于
generic_gcmaes_decrypt needs to use generic_gcmaes_ctx, not aesni_rfc4106_gcm_ctx. This is actually harmless because the fields in struct generic_gcmaes_ctx share the layout of the same fields in aesni_rfc4106_gcm_ctx. Fixes: cce2ea8d ("crypto: aesni - add generic gcm(aes)") Cc: <stable@vger.kernel.org> Signed-off-by: NSabrina Dubroca <sd@queasysnail.net> Reviewed-by: NStefano Brivio <sbrivio@redhat.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 22 9月, 2017 2 次提交
-
-
由 Corentin LABBE 提交于
This patch replace GCM IV size value by their constant name. Signed-off-by: NCorentin Labbe <clabbe.montjoie@gmail.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
由 Colin Ian King 提交于
Arrays aesni_simd_skciphers and aesni_simd_skciphers2 are local to the source and do not need to be in global scope, so make them static. Cleans up sparse warnings: symbol 'aesni_simd_skciphers' was not declared. Should it be static? symbol 'aesni_simd_skciphers2' was not declared. Should it be static? Signed-off-by: NColin Ian King <colin.king@canonical.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 04 8月, 2017 1 次提交
-
-
由 Ard Biesheuvel 提交于
There are quite a number of occurrences in the kernel of the pattern if (dst != src) memcpy(dst, src, walk.total % AES_BLOCK_SIZE); crypto_xor(dst, final, walk.total % AES_BLOCK_SIZE); or crypto_xor(keystream, src, nbytes); memcpy(dst, keystream, nbytes); where crypto_xor() is preceded or followed by a memcpy() invocation that is only there because crypto_xor() uses its output parameter as one of the inputs. To avoid having to add new instances of this pattern in the arm64 code, which will be refactored to implement non-SIMD fallbacks, add an alternative implementation called crypto_xor_cpy(), taking separate input and output arguments. This removes the need for the separate memcpy(). Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 18 5月, 2017 1 次提交
-
-
由 Sabrina Dubroca 提交于
Now that the asm side of things can support all the valid lengths of ICV and all lengths of associated data, provide the glue code to expose a generic gcm(aes) crypto algorithm. Signed-off-by: NSabrina Dubroca <sd@queasysnail.net> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 03 2月, 2017 1 次提交
-
-
由 Herbert Xu 提交于
When aesni is built as a module together with pcbc, the pcbc module must be present for aesni to load. However, the pcbc module may not be present for reasons such as its absence on initramfs. This patch allows the aesni to function even if the pcbc module is enabled but not present. Reported-by: NArkadiusz Miśkiewicz <arekm@maven.pl> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 30 12月, 2016 1 次提交
-
-
由 Herbert Xu 提交于
If aesni is built-in but pcbc is built as a module, then aesni will fail completely because when it tries to register the pcbc variant of aes the pcbc template is not available. This patch fixes this by modifying the pcbc presence test so that if aesni is built-in then pcbc must also be built-in for it to be used by aesni. Fixes: 85671860 ("crypto: aesni - Convert to skcipher") Reported-by: NStephan Müller <smueller@chronox.de> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 27 12月, 2016 1 次提交
-
-
由 Ilya Lesokhin 提交于
In the common case of !PageHighMem we can do zero copy crypto even if sg crosses a pages boundary. Signed-off-by: NIlya Lesokhin <ilyal@mellanox.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 28 11月, 2016 1 次提交
-
-
由 Herbert Xu 提交于
This patch converts aesni (including fpu) over to the skcipher interface. The LRW implementation has been removed as the generic LRW code can now be used directly on top of the accelerated ECB implementation. Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 12 11月, 2016 1 次提交
-
-
由 Arnd Bergmann 提交于
The rfc4106 encrypy/decrypt helper functions cause an annoying false-positive warning in allmodconfig if we turn on -Wmaybe-uninitialized warnings again: arch/x86/crypto/aesni-intel_glue.c: In function ‘helper_rfc4106_decrypt’: include/linux/scatterlist.h:67:31: warning: ‘dst_sg_walk.sg’ may be used uninitialized in this function [-Wmaybe-uninitialized] The problem seems to be that the compiler doesn't track the state of the 'one_entry_in_sg' variable across the kernel_fpu_begin/kernel_fpu_end section. This takes the easy way out by adding a bogus initialization, which should be harmless enough to get the patch into v4.9 so we can turn on this warning again by default without producing useless output. A follow-up patch for v4.10 rearranges the code to make the warning go away. Signed-off-by: NArnd Bergmann <arnd@arndb.de> Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
-
- 01 7月, 2016 1 次提交
-
-
由 Herbert Xu 提交于
Currently aesni uses an async ctr(aes) to derive the rfc4106 subkey, which was presumably copied over from the generic rfc4106 code. Over there it's done that way because we already have a ctr(aes) spawn. But it is simply overkill for aesni since we have to go get a ctr(aes) from scratch anyway. This patch simplifies the subkey derivation by using a straight aes cipher instead. Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 23 6月, 2016 1 次提交
-
-
由 Herbert Xu 提交于
This patch fixes an old bug where gcm requests can be reordered because some are processed by cryptd while others are processed directly in softirq context. The fix is to always postpone to cryptd if there are currently requests outstanding from the same tfm. Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 13 4月, 2016 1 次提交
-
-
由 Borislav Petkov 提交于
Signed-off-by: NBorislav Petkov <bp@suse.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-crypto@vger.kernel.org Link: http://lkml.kernel.org/r/1459801503-15600-4-git-send-email-bp@alien8.deSigned-off-by: NIngo Molnar <mingo@kernel.org>
-
- 17 2月, 2016 1 次提交
-
-
由 Stephan Mueller 提交于
The patch centralizes the XTS key check logic into the service function xts_check_key which is invoked from the different XTS implementations. With this, the XTS implementations in ARM, ARM64, PPC and S390 have now a sanity check for the XTS keys similar to the other arches. In addition, this service function received a check to ensure that the key != the tweak key which is mandated by FIPS 140-2 IG A.9. As the check is not present in the standards defining XTS, it is only enforced in FIPS mode of the kernel. Signed-off-by: NStephan Mueller <smueller@chronox.de> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 17 8月, 2015 1 次提交
-
-
由 Herbert Xu 提交于
This patch removes the CRYPTO_ALG_AEAD_NEW flag now that everyone has been converted. Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 14 7月, 2015 1 次提交
-
-
由 Herbert Xu 提交于
This patch converts rfc4106 to the new calling convention where the IV is now in the AD and needs to be skipped. Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 29 6月, 2015 1 次提交
-
-
由 Tadeusz Struk 提交于
rfc4106(gcm(aes)) uses ctr(aes) to generate hash key. ctr(aes) needs chainiv, but the chainiv gets initialized after aesni_intel when both are statically linked so the setkey fails. This patch forces aesni_intel to be initialized after chainiv. Signed-off-by: NTadeusz Struk <tadeusz.struk@intel.com> Tested-by: NLinus Torvalds <torvalds@linux-foundation.org> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 03 6月, 2015 2 次提交
-
-
由 Herbert Xu 提交于
This patch converts the low-level __gcm-aes-aesni algorithm to the new AEAD interface. Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
由 Herbert Xu 提交于
This patch converts rfc4106-gcm-aesni to the new AEAD interface. The low-level interface remains as is for now because we can't touch it until cryptd itself is upgraded. In the conversion I've also removed the duplicate copy of the context in the top-level algorithm. Now all processing is carried out in the low-level __driver-gcm-aes-aesni algorithm. Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 19 5月, 2015 1 次提交
-
-
由 Ingo Molnar 提交于
We already have fpu/types.h, move i387.h to fpu/api.h. The file name has become a misnomer anyway: it offers generic FPU APIs, but is not limited to i387 functionality. Reviewed-by: NBorislav Petkov <bp@alien8.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: NIngo Molnar <mingo@kernel.org>
-
- 13 5月, 2015 1 次提交
-
-
由 Herbert Xu 提交于
This patch uses the crypto_aead_set_reqsize helper to avoid directly touching the internals of aead. Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 31 3月, 2015 1 次提交
-
-
由 Stephan Mueller 提交于
Flag all AES-NI helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: NStephan Mueller <smueller@chronox.de> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 13 3月, 2015 1 次提交
-
-
由 Stephan Mueller 提交于
The kernel crypto API logic requires the caller to provide the length of (ciphertext || authentication tag) as cryptlen for the AEAD decryption operation. Thus, the cipher implementation must calculate the size of the plaintext output itself and cannot simply use cryptlen. The RFC4106 GCM decryption operation tries to overwrite cryptlen memory in req->dst. As the destination buffer for decryption only needs to hold the plaintext memory but cryptlen references the input buffer holding (ciphertext || authentication tag), the assumption of the destination buffer length in RFC4106 GCM operation leads to a too large size. This patch simply uses the already calculated plaintext size. In addition, this patch fixes the offset calculation of the AAD buffer pointer: as mentioned before, cryptlen already includes the size of the tag. Thus, the tag does not need to be added. With the addition, the AAD will be written beyond the already allocated buffer. Note, this fixes a kernel crash that can be triggered from user space via AF_ALG(aead) -- simply use the libkcapi test application from [1] and update it to use rfc4106-gcm-aes. Using [1], the changes were tested using CAVS vectors to demonstrate that the crypto operation still delivers the right results. [1] http://www.chronox.de/libkcapi.html CC: Tadeusz Struk <tadeusz.struk@intel.com> Cc: stable@vger.kernel.org Signed-off-by: NStephan Mueller <smueller@chronox.de> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 28 2月, 2015 1 次提交
-
-
由 Tadeusz Struk 提交于
Changed the __driver-gcm-aes-aesni to be a proper aead algorithm. This required a valid setkey and setauthsize functions to be added and also some changes to make sure that math context is not corrupted when the alg is used directly. Note that the __driver-gcm-aes-aesni should not be used directly by modules that can use it in interrupt context as we don't have a good fallback mechanism in this case. Signed-off-by: NAdrian Hoban <adrian.hoban@intel.com> Signed-off-by: NTadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 14 1月, 2015 1 次提交
-
-
由 Timothy McCaffrey 提交于
These patches fix the RFC4106 implementation in the aesni-intel module so it supports 192 & 256 bit keys. Since the AVX support that was added to this module also only supports 128 bit keys, and this patch only affects the SSE implementation, changes were also made to use the SSE version if key sizes other than 128 are specified. RFC4106 specifies that 192 & 256 bit keys must be supported (section 8.4). Also, this should fix Strongswan issue 341 where the aesni module needs to be unloaded if 256 bit keys are used: http://wiki.strongswan.org/issues/341 This patch has been tested with Sandy Bridge and Haswell processors. With 128 bit keys and input buffers > 512 bytes a slight performance degradation was noticed (~1%). For input buffers of less than 512 bytes there was no performance impact. Compared to 128 bit keys, 256 bit key size performance is approx. .5 cycles per byte slower on Sandy Bridge, and .37 cycles per byte slower on Haswell (vs. SSE code). This patch has also been tested with StrongSwan IPSec connections where it worked correctly. I created this diff from a git clone of crypto-2.6.git. Any questions, please feel free to contact me. Signed-off-by: NTimothy McCaffrey <timothy.mccaffrey@unisys.com> Signed-off-by: NJarod Wilson <jarod@redhat.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 24 11月, 2014 1 次提交
-
-
由 Kees Cook 提交于
This prefixes all crypto module loading with "crypto-" so we never run the risk of exposing module auto-loading to userspace via a crypto API, as demonstrated by Mathias Krause: https://lkml.org/lkml/2013/3/4/70Signed-off-by: NKees Cook <keescook@chromium.org> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 06 11月, 2014 1 次提交
-
-
由 Valentin Rothberg 提交于
The CPP identifier 'HAS_PCBC' is defined when the Kconfig option CRYPTO_PCBC is set as 'y' or 'm', and is further used in two ifdef blocks to conditionally compile source code. This indirection hides the actual Kconfig dependency and complicates readability. Moreover, it's inconsistent with the rest of the ifdef blocks in the file, which directly reference Kconfig options. This patch removes 'HAS_PCBC' and replaces its occurrences with the actual dependency on 'CRYPTO_PCBC' being set as 'y' or 'm'. Signed-off-by: NValentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 02 10月, 2014 1 次提交
-
-
由 Mathias Krause 提交于
This reverts commit 7da4b29d. Now, that the issue is fixed, we can re-enable the code. Signed-off-by: NMathias Krause <minipli@googlemail.com> Cc: Chandramouli Narayanan <mouli@linux.intel.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-
- 24 9月, 2014 1 次提交
-
-
由 Mathias Krause 提交于
The "by8" implementation introduced in commit 22cddcc7 ("crypto: aes - AES CTR x86_64 "by8" AVX optimization") is failing crypto tests as it handles counter block overflows differently. It only accounts the right most 32 bit as a counter -- not the whole block as all other implementations do. This makes it fail the cryptomgr test #4 that specifically tests this corner case. As we're quite late in the release cycle, just disable the "by8" variant for now. Reported-by: NRomain Francoise <romain@orebokech.com> Signed-off-by: NMathias Krause <minipli@googlemail.com> Cc: Chandramouli Narayanan <mouli@linux.intel.com> Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
-