You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TokenList::Stream turns the input bytes into characters: it decodes UTF-16, and it turns CR and CR LF into LF. Putting a character
back (ungetChar()) is passed down to the stream classes as raw byte unget() calls:
Class
unget()
StdIStream
istream::unget()
StdCharBufStream
--pos
FileStream
ungetc(), or fseek() for UTF-16
So only one character can safely be put back, and ungetChar() must undo the UTF-16 decoding byte by byte (C only guarantees one
character of ungetc() pushback).
Step 2 of #708 moves line splicing into the stream. Then readChar() must look past \, any spaces, and an optional CR to see if a
newline follows. If it doesn't, all of those characters must be put back.
Add a small buffer of already decoded characters to Stream:
readChar() and peekChar() take characters from the buffer first.
ungetChar() puts the last character back into the buffer, instead of calling unget() on the stream classes.
More than one character can be put back.
files with CR-only line endings are misread with file input, and a CR at the end of the file is not treated as a newline.
Activity
[-]Refactoring: add a lookahead buffer to TokenList::Stream[/-][+]Add a lookahead buffer to TokenList::Stream[/+]Fix cppcheck-opensource#739 (Add a lookahead buffer to TokenList::Str…