summaryrefslogtreecommitdiff
path: root/src/buffer.c
AgeCommit message (Collapse)Author
2016-06-06msvc: Fix warnings and errorsVicent Marti
2016-06-06mem: Add a `realloc` pointer to the memory handlerVicent Marti
2016-06-06buffer: rever to using a 32-bit bufsize_tVicent Marti
2016-06-06cmark: Implement support for custom allocatorsVicent Marti
2016-06-06cmake: Global handler for OOM situationsVicent Marti
2016-06-06buffer: proper safety checks for unbounded memoryVicent Marti
The previous work for unbounded memory usage and overflows on the buffer API had several shortcomings: 1. The total size of the buffer was limited by arbitrarily small precision on the storage type for buffer indexes (typedef'd as `bufsize_t`). This is not a good design pattern in secure applications, particualarly since it requires the addition of helper functions to cast to/from the native `size` types and the custom type for the buffer, and check for overflows. 2. The library was calling `abort` on overflow and memory allocation failures. This is not a good practice for production libraries, since it turns a potential RCE into a trivial, guaranteed DoS to the whole application that is linked against the library. It defeats the whole point of performing overflow or allocation checks when the checks will crash the library and the enclosing program anyway. 3. The default size limits for buffers were essentially unbounded (capped to the precision of the storage type) and could lead to DoS attacks by simple memory exhaustion (particularly critical in 32-bit platforms). This is not a good practice for a library that handles arbitrary user input. Hence, this patchset provides slight (but in my opinion critical) improvements on this area, copying some of the patterns we've used in the past for high throughput, security sensitive Markdown parsers: 1. The storage type for buffer sizes is now platform native (`ssize_t`). Ideally, this would be a `size_t`, but several parts of the code expect buffer indexes to be possibly negative. Either way, switching to a `size` type is an strict improvement, particularly in 64-bit platforms. All the helpers that assured that values cannot escape the `size` range have been removed, since they are superfluous. 2. The overflow checks have been removed. Instead, the maximum size for a buffer has been set to a safe value for production usage (32mb) that can be proven not to overflow in practice. Users that need to parse particularly large Markdown documents can increase this value. A static, compile-time check has been added to ensure that the maximum buffer size cannot overflow on any growth operations. 3. The library no longer aborts on buffer overflow. The CMark library now follows the convention of other Markdown implementations (such as Hoedown and Sundown) and silently handles buffer overflows and allocation failures by dropping data from the buffer. The result is that pathological Markdown documents that try to exploit the library will instead generate truncated (but valid, and safe) outputs. All tests after these small refactorings have been verified to pass. --- NOTE: Regarding 32 bit overflows, generating test cases that crash the library is trivial (any input document larger than 2gb will crash CMark), but most Python implementations have issues with large strings to begin with, so a test case cannot be added to the pathological tests suite, since it's written in Python.
2016-03-12Switch from "inline" to "CMARK_INLINE"Nick Wellnhofer
Newer MSVC versions support enough of C99 to be able to compile cmark in plain C mode. Only the "inline" keyword is still unsupported. We have to use "__inline" instead.
2016-01-22Removed an unnecessary C99-ism in buffer.c.John MacFarlane
This helps compiling on systems like luarocks that don't have all the cmake configuration goodness. Thanks to @carlmartus
2015-08-10Remove need to disable MSVC warning 4244Kevin Wojniak
2015-08-08Improved cmark_strbuf_normalize_whitespace.John MacFarlane
Now all characters that satisfy cmark_isspace are recognized as whitespace. Previously CR and TAB (and others) weren't included. Partially addresses #73.
2015-07-27Use clang-format, llvm style, for formatting.John MacFarlane
* Reformatted all source files. * Added 'format' target to Makefile. * Removed 'astyle' target. * Updated .editorconfig.
2015-07-25Removed cmark_strbuf_printf and cmark_strbuf_vprintf.John MacFarlane
These are no longer needed, and cause complications for MSVC. Also removed HAVE_VA_COPY and HAVE_C99_SNPRINTF feature tests.
2015-06-16astyle formatting changes.John MacFarlane
2015-06-07Helper to safely call strlenNick Wellnhofer
2015-06-07Change return type of cmark_strbuf_lenNick Wellnhofer
2015-06-07Missing bounds checks in buffer.cNick Wellnhofer
2015-06-07Remove unused function cmark_strbuf_attachNick Wellnhofer
This function was missing a couple of range checks that I'm too lazy to fix.
2015-06-07Fix check in cmark_strbuf_vprintfNick Wellnhofer
Avoid potential overflow and allow for different bufsize types.
2015-06-07Check for negative lengths in buffer.cNick Wellnhofer
2015-06-07Check for overflow when growing strbufsNick Wellnhofer
Replace macro ENSURE_SIZE with inline function S_strbuf_grow_by that checks for overflow.
2015-06-07Remove useless code in cmark_strbuf_growNick Wellnhofer
cmark_strbuf_grow will never truncate a buffer.
2015-06-07Account for null terminator in cmark_strbuf_growNick Wellnhofer
This simplifies overflow checks.
2015-06-07Check for overflow in cmark_strbuf_growNick Wellnhofer
2015-06-07Simplify oversizing of strbufsNick Wellnhofer
Always add 50% on top of target size. No need for a loop.
2015-06-07Use custom type bufsize_t for string buffer sizesNick Wellnhofer
This makes it easier to change the type later. No functional change. The rest of the code base still has to be adjusted to use the new type. Also add some TODO comments in buffer.c.
2015-06-07Abort on strbuf errorsNick Wellnhofer
Users of the strbuf API are supposed to check for an OOM condition after appending to strbufs, but: * This is never done in the whole code base. * The implementation was flawed because only `ptr` was set to the OOM value without adjusting `size` and `asize`. After an error, subsequent calls could very well lead to segfaults, contrary to the documentation. Change the code to always abort on errors with a message printed to stderr. The only alternative is to propagate errors throughout the whole library which seems infeasible.
2015-06-06fix ENSURE_SIZE to actually check left arg length.bryant
2015-05-29Cope with broken snprintf on WindowsNick Wellnhofer
On Windows, snprintf returns -1 if the output was truncated. Fall back to Windows-specific _scprintf.
2015-03-29Fixed bug in cmark_strbuf_unescape (buffer.c).John MacFarlane
The old function called 'continue' when seeing a backslash, but this gave incorrect results on input like: \\* since the next backslash would be treated as escaping the `*` instead of being escaped itself.
2015-01-05Reformatted code consistently with astyle.John MacFarlane
2014-12-31Feature test for va_copyNick Wellnhofer
MSVC doesn't support va_copy.
2014-12-31Add missing va_endNick Wellnhofer
2014-12-30Revert "Remove unneeded va_copy"John MacFarlane
This reverts commit 485ef21b95e257e9d9cbcaa804c3c164f1f49a80. Apparently the va_copy IS needed, because without this code we get segfaults in some cases. Closes #253. @nwellnhof, can you have a look at this issue and comment? I understand that this code was removed for portability reasons. Is there an alternative solution?
2014-12-29Added cmark_ prefix to functions in cmark_ctype.John MacFarlane
2014-12-29Added cmark_ctype.h with locale-independent isspace, ispunct, etc.John MacFarlane
Otherwise cmark's behavior varies unpredictably with the locale. `is_punctuation` in utf8.h has also been adjusted so that everything that counts all ASCII symbol characters count as punctuation, even though some are not in P* character classes.
2014-12-15Re-added cmark_ prefix to strbuf and chunk.John MacFarlane
Reverts 225d720.
2014-12-13Removed cmark_ prefix on chunk and strbuf.John MacFarlane
This isn't needed any more since we don't expose these in the API.
2014-11-16Move inline function definitions to header filesNick Wellnhofer
Inline functions must be defined in header files in order to be inlined in other compilation units. This also fixes the MSVC build where out-of-line versions weren't created and allows to remove the -fgnu89-inline flag.
2014-11-16Remove unneeded va_copyNick Wellnhofer
va_copy isn't needed here. See http://stackoverflow.com/questions/26953289 Remove it because it isn't part of C89 and not implemented by MSVC.
2014-11-16Remove unneeded #includesNick Wellnhofer
Fixes cross-platform issues.
2014-11-16Cast void pointers explicitlyNick Wellnhofer
Needed for C++ compatibility.
2014-11-15Expose lower-level parsing API.John MacFarlane
The new functions cmark_new_doc_parser, cmark_free_doc_parser, cmark_process_line, and cmark_finish allow you to feed lines one by one (possibly from several files) to the parser and call finish when you're done. This is now used in main for mulitple files.
2014-11-14Moved code from buffer.h to buffer.c. Make strbuf_cstr static.John MacFarlane
2014-11-14buffer: Name functions with cmark_ prefix.John MacFarlane
2014-11-06buffer: added strbuf_rtrim function.John MacFarlane
2014-10-24Only define MIN if it's undefined.John MacFarlane
Note: it is defined in GNU libc (Linux) and FreeBSD and OSX sys/param.h.
2014-10-24Merge branch 'master' of https://github.com/abduelhamit/stmd into ↵John MacFarlane
abduelhamit-master
2014-10-18Reindented c sources.John MacFarlane
2014-10-13Add missing MIN directiveAbdülhamit Yilmaz
It's needed in strbuf_cmp().
2014-09-10Do not create references with empty namesVicent Marti