csv module documentation: some redundant statement

[alextretyak]

From here:

If csvfile is a file object, it should be opened with newline=''.

It looks like this statement is useless/redundant for csv.reader (i.e. it is useful only for csv.writer), as newline=None [by default] works fine on any platform.

Linked PRs

• gh-108838: Improve csv documentation #108850
• gh-108838: Improve docs - remove redundant newline statement for csv reader docs #110256

[serhiy-storchaka]

It is not useless, because with newline=None and newline='' you can get different results if the input contains a multiline quoted field. For example:

"a
b"

[alextretyak]

Can you give some working example?
I've checked the case with multiline quoted fields on both Windows and on Linux, and I haven't seen any different results between newline=None and newline=''.

When newline is None, universal newlines mode is enabled at file level, but it does not bother with the csv module own (universal) newline handling.

[alextretyak]

If the csv file contains (file was saved with Windows-style line endings):

"a
b"

I always get

['a\nb']

[serhiy-storchaka]

>>> import csv, io
>>> next(csv.reader(io.TextIOWrapper(io.BytesIO(b'"a\r\nb"'), newline='')))
['a\r\nb']
>>> next(csv.reader(io.TextIOWrapper(io.BytesIO(b'"a\r\nb"'), newline=None)))
['a\nb']

[alextretyak]

>>> next(csv.reader(io.TextIOWrapper(io.BytesIO(b'"a\r\nb"'), newline='')))
['a\r\nb']

This is not correct behaviour, I think. As \r character is not expected inside a string in any practical program. How should it [\r] be handled?
The sole purpose of the newline argument is to get rid of all \r characters correctly.

And to read csv from a string, code like this will be used:

next(csv.reader(io.StringIO('"a\nb"')))

[serhiy-storchaka]

RFC 4180, section 2.6:

   6.  Fields containing line breaks (CRLF), double quotes, and commas
       should be enclosed in double-quotes.  For example:

       "aaa","b CRLF
       bb","ccc" CRLF
       zzz,yyy,xxx

So \r\n is expected in a quoted field.

In any case, this issue is about the documentation. And the documentation is correct, requiring to open input files with newline=''.

[alextretyak]

RFC 4180

Please, do not confuse how data is transferred over the network with how it is stored in str variables in the programs.

For example, CRLF's are used in HTTP headers, but they never appear in strings:

from urllib import request
with request.urlopen('https://google.com') as f:
    print(repr(str(f.info()))) # print all HTTP headers (printed headers are delimited with LF, not CRLF!)

And the documentation is correct, requiring to open input files with newline=''.

No, it is not. If you open the input file via open() (as given in the usage example here), then there is no difference between newline=None and newline=''.

[serhiy-storchaka]

There is a difference. open() in text mode returns TextIOWrapper which handles newline translations. I used TextIOWrapper and BytesIO for simplicity, but here is an example with open():

>>> import csv
>>> with open('test.csv', 'w', newline='') as f:
...     csv.writer(f).writerow(['a\r\nb', 'a\nb', 'a\rb'])
... 
20
>>> with open('test.csv', 'r', newline='') as f:
...     print(next(csv.reader(f)))
... 
['a\r\nb', 'a\nb', 'a\rb']
>>> with open('test.csv', 'r', newline=None) as f:
...     print(next(csv.reader(f)))
... 
['a\nb', 'a\nb', 'a\nb']

[alextretyak]

Hm. It looks like I mistested this case.
But then it's even worse: the csv reader leaves CR's inside the strings for csv files with multiline records stored with Windows-style line endings. Although this is a subject for another issue...