Messages in this thread | | | Date | Thu, 25 Jul 2013 13:29:04 +0200 | Subject | Large pastes into readline enabled programs causes breakage from v2.6.31 onwards | From | Margarita Manterola <> |
| |
Hi,
The problem: Large pastes (5k or more) into a readline enabled program fail when running kernels larger than v2.6.31-rc5. "Fail" means that some lines are incomplete. From v2.6.39-rc1 onwards, "some lines" become "almost all lines after the first 4k". This turns up at least in Fedora, Debian, Ubuntu and Gentoo. From our findings, it should happen in any readline enabled program on a system running kernels later than the mentioned ones.
The problematic commits in the kernel tree: 1 - 2009-07-27 (never shipped) - https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=3a54297478e6578f96fd54bf4daa1751130aca86
After this commit, pastes start breaking. For a 35k file, about 50% of the times one or two lines are partially incomplete.
2 - 2009-07-29 (v2.6.31-rc5) - https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=e043e42bdb66885b3ac10d27a01ccb9972e2b0a3
This commit reverts the previous one, but adds one extra call to flush_to_ldisc. Pastes still break, commenting out the function call prevents breakage *up to 2.6.39-rc1*.
3 - 2011-03-22 (v2.6.39-rc1) - https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=f23eb2b2b28547fc70df82dd5049eb39bec5ba12
This commit changes many schedule/flush/cancel_delayed_work calls into schedule/flush/cancel_work. After this commit, the big breakage starts: for the 35k example file, it starts breaking at aprox. 4k and then every line is partially incomplete or directly not there.
Still after this commit, commenting out the tty_flush_to_ldisc(tty) call added by e043e42bdb66885b3ac10d27a01ccb9972e2b0a3 prevents the breakage.
4 - 2011-04-04 (v2.6.39-rc2) - https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=a5660b41af6a28f8004e70eb261e1202ad55c5e3
This commit modifies the behaviour of how the ttys are polled. After this commit, commenting out the tty_flush_to_ldisc(tty) call doesn't prevent breakage anymore.
But then re-adding the call to schedule_work(&tty->buf.work) that was removed in this commit, prevents the breakage even up to 3.11-rc2. I'm attaching a diff of the patch that we applied, just to show what had to be done, this is not a proposed fix, because this does cause a busy loop that is particularly noticeable in VMs.
We haven't yet found a good fix for this issue, but we believe that pasting into readline enabled programs shouldn't cause characters to get lost, and it should be possible to do that properly without the busy loop.
*** This was originally reported as a bug in readline, but it was found that going back to very old kernels prevented the issue, regardless of the version of readline.
Original Report (2012-06-25): http://lists.gnu.org/archive/html/bug-readline/2012-06/msg00006.html Follow Up thread (2013-07-22): http://lists.gnu.org/archive/html/bug-readline/2013-07/msg00006.html
I'm attaching here a very simple readline enabled program that helps with performing tests. Compile, run, then copy and paste a large enough file into it, close and diff.
Looking at the code in readline, the issue is triggered by these lines in rltty.c, while preparing the input:
tiop->c_lflag &= ~(ICANON | ECHO); (...) tiop->c_iflag &= ~(ICRNL | INLCR);
If those two lines are replaced by:
tiop->c_lflag &= ~(ECHO); (...) tiop->c_iflag &= ~(INLCR);
Then the pastes work fine: no lines are missing. Of course, this means that readline doesn't work properly, but this is just to note that those are the terminal settings that cause the issue to pop-up.
Credit: this investigation was done together with Maximiliano Curia.
-- Regards, Margarita Manterola [unhandled content-type:application/octet-stream] | |