rslib: Allocate decoder buffers to avoid VLAs
authorThomas Gleixner <tglx@linutronix.de>
Sun, 22 Apr 2018 16:23:55 +0000 (18:23 +0200)
committerKees Cook <keescook@chromium.org>
Wed, 25 Apr 2018 02:50:10 +0000 (19:50 -0700)
commit45888b40d2a6221d46bb69959e2600ddba71cc1f
tree58422ab63f1bcc01b810cb2c7d2a8c7be0fd2bb0
parent964dfce9c2b323a9a9d0bd6764e0f530b40104e4
rslib: Allocate decoder buffers to avoid VLAs

To get rid of the variable length arrays on stack in the RS decoder it's
necessary to allocate the decoder buffers per control structure instance.

All usage sites have been checked for potential parallel decoder usage and
fixed where necessary. Kees confirmed that the pstore decoding is strictly
single threaded so there should be no surprises.

Allocate them in the rs control structure sized depending on the number of
roots for the chosen codec and adapt the decoder code to make use of them.

Document the fact that decode operations based on a particular rs control
instance cannot run in parallel and the caller has to ensure that as it's
not possible to provide a proper locking construct which fits all use
cases.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Kees Cook <keescook@chromium.org>
Cc: Boris Brezillon <boris.brezillon@free-electrons.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Segher Boessenkool <segher@kernel.crashing.org>
Cc: Kernel Hardening <kernel-hardening@lists.openwall.com>
Cc: Richard Weinberger <richard@nod.at>
Cc: Mike Snitzer <snitzer@redhat.com>
Cc: Anton Vorontsov <anton@enomsg.org>
Cc: Colin Cross <ccross@android.com>
Cc: Andrew Morton <akpm@linuxfoundation.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Alasdair Kergon <agk@redhat.com>
Signed-off-by: Kees Cook <keescook@chromium.org>
include/linux/rslib.h
lib/reed_solomon/decode_rs.c
lib/reed_solomon/reed_solomon.c