Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 122815
|
Description
Robert Scheck
2004-05-08 14:45:50 UTC
It seems so, that the problem came up after ncurses-5.3-9.3 (RHEL 3). Sorry, I don't have more packages between 5.3-9.3 and 5.4-4.1 to trigger out the problem exactly. *** Bug 124039 has been marked as a duplicate of this bug. *** This is a particularly annoying bug. Affects mutt, nano, less... This problem is caused by the so called fix for bug 115448. I must say that I'm shocked by the logic in this bug report: * hey something broke colors in some apps * hmm setting TERM to gnome-rh9 fixes this * bla * letts hardcode a link of TERM=xterm to TERM=gnome-rh9 in terminfo me: * GASP * Hello, people do use other terminal emulators to. I believe atleast konsole also is officially supported by Fedora, we also have the real xterm and others. Besides that people who use a remote terminal connection to a FC2 system might also appreciate a somewhat sane default xterm terminfo entry. This leads me to 2 possible terminfo fixes, first the preferred one -add an xterm-redhat (like there still is in termcap, and like there used to be, which has always worked fine): *which links to xterm-xfree86 (which should also fix the vim problem) *which adds the RH/debian kbguidelines mappings for home and end -make xterm point to xterm-xfree86 Or atleast fix home and end by adding a mapping for home and end to the gnome terminfo entry which for some reason doesn't have any entries for the home and end keys, and seems broken in general. I'm going to write patches for both these fixes right now and will attach those later today. BTW next time something like this breaks please try to look bakc to what might have caused the breakage and undo this, things like this are very delicate and the old solution is very well tested. * AAARGGGHHHH * I just tested konsole and gnome-terminal and they don't send the redhat/debian kb guideline sequences for home and end anymore but instead send other ESC sequences. I now have 3 xterm clone temrinal emulators: -xterm -gnome-terminal -konsole Which each send different esc-sequences for home and end, so fixing terminfo is impossible since there is NO default/correct behaviour to put in terminfo. I've been working with RH to get this right since 6.2 and it used to work fine, why o why have all the patches for this been blindly dropped when upgrading to newer upstream versions without checking why those patches where there in the first place? I've decided to skip one (ONE!) beta cycle and everything in the textmode/xterm keybindings world is messed up again. Sorry but this really frustrates me. In order to fix this we need to: -decide what the correct esc-sequences for home and end under a terminal claiming to be xterm are. -decide to which terminfo entry xterm should point (preferably xterm-xfree86. -fix terminfo, gnome-terminal, konsole and xterm. I'm going to assume that rh ones FC2 to be compatible with rh 6.2-fc1 (wenn using a remote terminal to/from these) and still wishes to follow to debiankeyboard guidelines (for the esc sequences for home and end, which was decided in the 6.2 days). So I'll create a bunch of patches to put this all back to the way it used to be and it used to work. I'll submit the gnome-terminal and konsole patches under a new bug, pointing to this bug for the explanation. For those wanting to know more on this just search on bugs submitted by me, I've already explained this too many times. Created attachment 100583 [details]
patch fixing the problem
Replace the 5.4-color_xterm patch by this one and you will atleast have an
entry in the xterm terminfo for home and end, this entry currently contains the
esc-sequence as used in the past, meaning that it will only work in xterm.
I'll produce patches for gnoem-terminal and konsole to fix this. (sigh)
In the meantime to see what this patch does try: "tput khome; echo $?" before
and after this patch as you will see $? is 1 before this patch because
currently there is NO definition for khome (and kend) in terminfo's xtemr
entry.
Well, gnome-terminal (vte) seems to determine the ESC-sequences for home and end by reading them from ncurses, so fixing ncurses also fixes gnome-terminal, hurray! ncurses rebuilt with Hans's updated -color_xterm patch fixes all my HOME/END problems in gnome-terminal using nano/mutt/less. Thanks Hans! Adrian, can this get built as an FC2 update please? Thank you very much, Hans, your patch solves my issue! :) Now, I only can add me to Joe's request...can we get an official update, Adrian? Hmmm, I'm seeing issues using mutt in gnome-terminal: open a long message, hit PgDown twice, and the display is corrupted with characters left over from previous screen, Ctrl-L is needed to display the page properly. This seems to be a regression since the FC2 ncurses, it goes away if I revert. That sounds like a problem reported elsewhere (one of the bugs in gnome-terminal). see for example http://bugs.gentoo.org/show_bug.cgi?id=43970 (I tried a while ago to subscribe, but that website is mildly broken). OK, yes, I do see that sometimes with the vanilla ncurses-5.4-5 packages too, so I guess it's not a regression in Hans' patch. Is this scrolling problem fixed in one of your patches for 5.4, do you know, Thomas? The real fix is to use the terminfo entry that I made for gnome-terminal (which doesn't use features that it doesn't implement correctly). I noted in another place that the problem is that gnome-terminal doesn't implement indn and rin properly. (That was on bug-ncurses, whose archive doesn't seem to be reachable - my email says it was on April 3, 2004). Hmm, ncurses-5.4-5 as shipped with FC2 contains an xterm terminfo entry which uses / is based on gnome-rh9. Thomas does that mean that the gnome-rh9 terminfo entry is NOT the terminfo entry that you made? Ifso where can I get your terminfo entry and what are the differences compared to plain xterm-xfree86 ? As promised I've created a patch fixing the home and end keys in konsole, see bug 124803 p.s. I think we should wait for Thomas before applying my fix to ncurses, if we can get a descent xterm terminfo entry which also works around some gnome-terminal bugs that would be even better. My suggestion is to use the xterm-xfree86 terminfo entry as a base but to leave out options not supported properly by gnome-terminal. That would be "gnome-rh90". Ignoring the function keys, there
are several interesting differences (glancing over the list,
they're things that are either not implemented in gnome-terminal,
or which it does incorrectly). Bear in mind that's from last
year - Fedora's gnome-terminal may have different problems.
For each line from infocmp, I could probably write a sentence
or two, but that takes time:
comparing gnome-rh90 to xterm-xfree86.
comparing booleans.
mc5i: F:T.
npc: F:T.
comparing numbers.
comparing strings.
blink: NULL, '\E[5m'.
cbt: NULL, '\E[Z'.
cnorm: '\E[?25h', '\E[?12l\E[?25h'.
cvvis: NULL, '\E[?12;25h'.
el1: NULL, '\E[1K'.
enacs: '\E)0', '\E(B\E)0'.
hpa: NULL, '\E[%i%p1%dG'.
ich: NULL, '\E[%p1%d@'.
indn: NULL, '\E[%p1%dS'.
invis: NULL, '\E[8m'.
is2: '\E7\E[r\E[m\E[?7h\E[?1;3;4;6l\E[4l\E8\E>', '\E[!p\E[?3;
4l\E[4l\E>>
mc0: NULL, '\E[i'.
mc4: NULL, '\E[4i'.
mc5: NULL, '\E[5i'.
rin: NULL, '\E[%p1%dT'.
rmcup: '\E[2J\E[?47l\E8', '\E[?1049l'.
rmso: '\E[m', '\E[27m'.
rmul: '\E[m', '\E[24m'.
rs1: NULL, '\Ec'.
rs2: '\E7\E[r\E8\E[m\E[?7h\E[?1;3;4;6l\E[4l\E>', '\E[!p\E[?3;
4l\E[4l\E>>
sgr: '\E[0%?%p6%t;1%;%?%p2%t;4%;%?%p1%p3%|%t;7%;m%?%p9%t\016%
e\017%;', >
sgr0: '\E[m', '\E[m\017'.
smcup: '\E7\E[?47h', '\E[?1049h'.
tbc: NULL, '\E[3g'.
vpa: NULL, '\E[%i%p1%dd'.
Hmm, My main (only) expertise when it comes to terminfo are keymappings, the other stuff I've never touched. So I could really use some help on this :) But I've always learned if something is worth doing it is worth doing well. Thomas, are you willing to help me and / or do you know a quick fix? My suggestion for a quick fix would be to use gnome-rh90 and fix the keys, but what kind of influence would this have for people using the real xterm (me), or using konsole (lots)? BTW what about Joe's comment that the corruption also happens with the vannilla ncurses-5.4-5, that suggests that using gnome-rh90 doesn't completly fix this. I'm not sure how to interpret Job's comment (it doesn't say what terminfo entry he was actually using). For the keys - Redhat has redefined most of xterm's function keys so they don't match the terminfo. Deleting most of their changes to the XTerm app-defaults file would improve things. It's sort of strange that gnome-terminal can copy the keyboard layout I made for xterm, but Redhat packager insists that users don't want it in xterm. About the scrolling problems Joe is seeing, I can't reproduce them with vim or irssi (both are mentioned on the gentoo bug link), I'll try with mutt next. I assume he is using TERM=xterm, with xterm pointing to gnome-rh90 (which it does in the vanilla FC2 ncurses-5.4-5) About the xterm function keys, this is an old patch of mine, which in those days used to be correct (make shift F1-F10 send F11-F20) back then gnome-terminal used todo this too. I've already submitted a patch to fix this see bug 100695. But unfortunatly this patch has been refused. yes - I noticed the 100695 report too late to respond (perhaps there's a way to have Redhat's bugzilla tell me about these as they're reported rather than require me to trawl for them). The unmodified xterm translations use the control-key modifier since 1996 when I added DECUDK support (DECUDK is documented to use the shift-key). My recollection is that this change came before Redhat's customization that uses the shift-key for keys F12-24 (or F10-F22). comment 7, comment 8: i've still testing the terminfo against the common list of "usual suspects" (especially important are the sshing/serial port terminals using the TERM=xterm case) also need to talk to nalin re current behavior in gnome-terminal *** Bug 123987 has been marked as a duplicate of this bug. *** I've successfully reproduced the problem Joe mentions in comment 9, and I've also written a fix for it by using xf86-v43 as the base for xterm-redhat (to which xterm then points) xf86-v43 doesn't use indn or rin which indeed seems to be the problem. This seems a correct way to fix things to me, since it basicly takes the xterm terminfo entry back to the way it was with FC1 Created attachment 100794 [details] Patch fixing the original problem and the bug reported in comment 9 This patch seems to hose line drawing with mutt through gnome-terminal. I'll need to look at it more carefully to see if it's salvagable, but in the meantime I can't take the patch as-is. Adrian, Do you mean my first or second patch, and can you give my a quick step by step on how to reproduce the problem? Maybe I should write a patch which just adds the home and end keys to the ncurses as currently shipped? I have an impression (from the other reports) that the line-drawing problem is a bug in slang. Just for amusement, you might try building mutt with ncursesw... *** Bug 125881 has been marked as a duplicate of this bug. *** Yet another duplicate, so it seems this bug is really anoying people, time to fix it. I now my fix has problems, I assume those line-drawing problems with mutt don't happen with the current release of ncurses. If that is the case I can create a fix which just adds home and end to the current terminfo xterm entry. Also I would still like a step-by-step on how to reproduce these problems. what's the status of fixing this annoying issue? The proposed fix is as follows: #1. revert the xterm definition back to the stock terminfo.src #2. Use new profile script to detect the two most popular color terminals (konsole and gnome-terminal), and set the TERM to either konsole or gnome appropriately, or to something else if COLORTERM is set to something wild. If COLORTERM is unset, set the TERM to something reasonable, like the latest xterm definition. Created attachment 101668 [details]
set TERM based on COLORTERM during profile startup
not sure why konsole sets COLORTERM but leaves it blank. "gnome" should
reference the latest gnome-terminal def. likewise for "konsole"... except the
konsole definition is only in later versions of ncurses; does not seem to be
present in RHEL3 (is present in FC). "xterm-new" should pull in the newest
xterm definition for xorg X-- and should be a relatively sane fallback when
sshing into a box.
Created attachment 101670 [details]
same as prev, except leave TERM alone if COLORTERM not set
remove the setting of TERM for the case of remote logins as per Katz's
suggestion on fedora-devel
Well Adrian, could you also do a offical rebuild for Fedora Core 2's update repository, if that solution is working fine (sorry, I didn't test it, yet)?! Would be nice - thanks :) comment 36: that's currently in progress (CLOSED RAWHIDE usually implies pushing to the repositories, which is done automatically for most FC* target collections a instance is built for) comment 36: i should add that the fix is two part; in addition to the new ncurses, you'll need the new setup (when it's built)-- or place the term.sh file in the patch section in /etc/profile.d There ara 1 big problem and two small problems with this fix: -the big problem is that it doesn't fix home and end, the original problem was that the xterm terminfo entry used the gnome terminfo entry which doesn't have home and end defined, now the xterm terminfo entry is ok, but the new term.sh makes $TERM be gnome for gnome-terminal, and the gnome terminfo entry still doesn't contain the ESC-sequences for home and end, I can write a patch to fix this. -the 1st small problem is that gnome-terminal in an attempt to get around problems with terminal versus terminfo mismatches, uses terminfo to decide which esc-sequences to use for certain keys, for example home and end. With the old ncurses, where xterm pointed to gnome, gnome-terminal couldn't get any esc-sequences for home and end (since the gnome terminfo entry doesn't have these) and used its default esc-sequences. With the new terminfo (ands setup.sh) it uses the esc-sequences which are defined in the xterm terminfo entry. So here we have a situation where gnome-terminal behaves accordingly to the xterm terminfo entry but $TERM is set to gnome. Now is that logical or what? -the 2nd small problem is that this fix is ugly as hell. As I've shown with my second patch it should be possible to create one simplified xterm terminfo entry which works for all terminals. My suggestions are: -tell me how the hell to reproduce the mutt line drawing problem and I'll look at it -or ask me to write a fix for the gnome terminfo entry defining home and end (and some other keys while I'm add it) -and incorperate this fix and make xterm point to gnome like it does in the current shipped ncurses, which doesn't seem to cause any problems except for home/end -or incorperate this fix and use term.sh So in order of (my) prefeference we've got: -still go for the default xterm entry for all terminals and look at the mutt problem. -fix the gnome terminfo entry and make xterm point to it -fix the gnome terminfo entry and use term.sh An advantage of the last fix is that konsole uses its own terminfo which gives possibilities to fix bug 124803 p.s. Since this isn't fixed it should be reopened but I haven't got the rights todo that. Also I read above that certain aspects of this bug we're discussed on fedora-devel, please don't do that what is the purpose of a bugtracking system if the decisions on how to fix the bug are made elsewhere? Reopening, because the latest proposed solution doesn't really solve the original problem, thank you Hans. Uh, setting TERM=gnome breaks just about every remote system I have access to. Solaris, HP-UX, FreeBSD, etc. systems do not know anything about it. Havill, Kaj has a very good point, that leaves the following 2 options 2 fix this: -go for the default xterm entry for all terminals and look at the mutt problem. (this requires being able to reproduce this problem) -fix the gnome terminfo entry and make it the default xterm entry I'm willing to write either fix. Just decide which one I should write. Lett me also point out that FC3 is coming soon and it would be very nice to have this fixed before FC3. Just make the call and I'll write the fix. Marked as FC3 target bug. Well, again, the latest available update packages don't really solve the issue for me, awaiting response from Adrian Havill... Kai would have a good point if gnome-terminal's functionality corresponded to the terminals on those other systems. Since it does not (and I, too frequently get complaints about the bugs in gnome-terminal), it would be a better solution if gnome-terminal identified itself properly. (If it's really supposed to be a supported and standard terminal, it should do that - as a minimum). Thomas, gnome-terminal tries to be xterm compatible, and sofar the only problem clearly defined seems to be the missing indn and rin capabilities, which according to terminfo are new in xterm xf4.4. Actually xterm also seems to not properly identify itself, since it also has had major changes lately and still calls itself xterm. So the question then becomes what should the default xterm terminfo entry represent, the old xterm-r6 ? Which should work everywhere, or the latest, greatest xterm-xfree which then breaks on older xterms and gnome-terms? I really don't want to have discussions about the principally best fix, and much rather have a pragmatic fix which works. So my proposal to fix this is to make the terminfo xterm entry represent the latest and greatest xterm as much as possible without breaking compatibility with older xterms and with xterm clones. On the other hand fixing bugs in xterm clones of course also can't hurt. Actually - though the terminfo for indn/rin is recent, the code in xterm to support it dates back several years (1996 or 1997). I added that because it made a nice test-case for ncurses. Offhand, I don't recall a recent (past 4-5 years, when I introduced the newer keyboard layout) change to the xterm terminfo which wouldn't work on the older versions of XFree86 xterm. Actually - though the terminfo for indn/rin is recent, the code in xterm to support it dates back several years (1996 or 1997). I added that because it made a nice test-case for ncurses. Offhand, I don't recall a recent (past 4-5 years, when I introduced the newer keyboard layout) change to the xterm terminfo which wouldn't work on the older versions of XFree86 xterm. comment 39: (addressing points in order) o the latest gnome-rh9 entry does re-add the home/end so that home/end work from gnome-terminal. i believe at the time you wrote that xterm was simply restored to it's original entry without modifications to the gnome-terminal o point 2 should be addressed by the additions of home/end in the terminfo for gnome-rh9 o i disagree with the attempt at creating a least-common-denominator version of the terminfo. this would only work if konsole/gnome-terminal were perfect supersets of xterm functionality, which they're not. until they are, the logical choice would be to accuratly identify the terminal and set TERM properly. comment 42: term.sh should only set TERM to gnome-terminal if it is in fact a gnome-terminal. sshing into a box should set TERM to xterm. comment 45: gome-terminal can be identified through the COLORTERM env var, which is what term.sh uses. If it sees a gnome-terminal identifier (and ONLY when it sees this), it sets TERM to gnome. It leaves TERM alone to the default value (xterm) if it senses neither a gnome-terminal nor a konsole. comment 46: re-iterating, i believe the best fix is to leave the terminfo.src alone as much as possible (with the exception of gnome-terminal, which according to comment 39, needs the home/end definitions... perhaps this fix belongs UPSTREAM in terminfo.src). Rather than trying to create one terminfo entry which pleases everyone, it seems more pragmatic and reliable to accurately identify when gnome-terminal and konsole are used, and set TERM appropriately. In all other cases TERM will be left alone. (term.sh actually doesn't belong in /etc/profile.d but later than this (/etc/bashrc perhaps... discoussing with notting), because the COLORTERM doesn't get set during the profile read, but after) *** Bug 127926 has been marked as a duplicate of this bug. *** Adrian, Forcing users to create workarounds to obtain desired functionality (eg. working termcap on another system) by having them set TERM=xterm when sshing from a gnome-terminal window to another system will just make FC3+ (and later RHEL4) stand out from the crowd in a negative way. People expect gnome-terminal to work out of the box like a. xterm or in some cases b. "the thing I access application X on server Y with". This also creates a problem for people who have, say, 5K users who suddenly start reporting degradation in functionality they expect. Please reconsider. :) I agree....as a user I should not have to do anything! If you can integrate your suggestion to all the applications that are effected by the above change so that the transition is seemless, than be my guest. BUT after installing new KDE packages I should not have to go into /etc/profile.d and change something. How many users know about environment variables. We keep going through this problem at least once a year: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=115448 either all the fedora/redhat community get together and resolve the problem or just keep patching Comment 51: I understand your point about "expect[ing] gnome-terminal to work out of the box like a. xterm", but this bug isn't about gnome-terminal's deficiencies-- it's about ncurses. I think the fairest thing to do is when TERM is set to "xterm", it really /is/ xterm. the advantage to keeping terminfo.src's xterm "pure" and using a terminal-detector such as term.sh that runs upon virtual terminal launch is as follows: o People using pure/true xterm connecting to fc/rhel boxes will get the pure xterm definition-- without any baggage/workarounds that may (but may not) interfere-- without setting any env vars. o People using gnome/konsole connecting to fc/rhel boxes will get a definition appropriate for their weird little terminal (that we've standardized on)-- without setting any env vars. The only disadvantage as far as I know: o for people sshing from gnome-terminal/konsole onto a box that DOES NOT use ncurses' terminfo and/or does not have a gnome/konsole entry-- these people will need to set their env var to "xterm" and file bugs against gnome-terminal/konsole when their term does not behave in a xterm compatible way. Comment 52: the only case where you would need to do anything would be is in the last case i mentioned above-- a gnome-term/konsole user sshing into a alien un*x box that doesn't use ncurses' termcap/terminfo. These people, and only these people, would need to change their TERM to xterm. you're right about hitting this problem again and again. it's time to stop kludging the terminfo to make up for gnome-terminal/konsole's differences/weaknesses (despite their strengths) there are too many terms out there that think they are "xterm" compatible that are going to be sshing into fc/rhel boxes. mucking with the default xterm definition is only going to get you into trouble with these people. People that want to run "gnome-terminal/konsole, etc." as a "xterm" and find problems when they do need to file a bug against their terminal, not against ncurses. I wish it was 100% compatible, but unfortunately, it's not-- and it's not ncurses' problem if the xterm definition is correct. Granted, it IS ncurses' problem if the definition for gnome-terminal is incorrect. That has been pointed out (the Home/End problem), and has been corrected. Adrian, Regarding Comment #53: if people have been using TERM=xterm in gnome-terminal and, for example, syntax coloring in vim they'll get ansi control codes on the screen with ncurses-5.4-10 while in ncurses-5.4-8 the screen looks all nice and dandy. I would be happy if xterm was synonymous with xterm-color.. OB-) comment 54: vim is an important application, but the solution again is not to change terminfo to compensate for an errant app-- vim should be fixed so that it behaves properly (vim has a lot of internal tricks and termpcap entries and it's trying to override/overcompensate). if we start getting near a code freeze point, i'll obviously need to unfortunately look at a hack as vim is too important to leave broken (even if it is vim's fault), but for now, I'm going to stay with my position that vim should be fixed to behave rather than terminfo be massaged to compensate for vim. Comment 49: -Fixing gnome-rh9 in terminfo (which at its own is a good thing todo) still leaves the problem that gnome uses xterm terminfo entry to determine which esq-sequences to send for home,end (and other keys) but now identifies itself to applications as gnome. -Comment 42 talksabout sshing out, not in Comment 53: You're hitting the nail right on its head here, there are going to be a lott of people using other os's / distros sshing into FC boxes, but this won't just work with the default terminfo xterm entry (see for example the vim bug, most of the other os's/distros wil also have gnome-terminal/konsole but these won't properly identify themselves, since they don't have term.sh. And even if they do get term.sh / or gnome-terminal is "fixed" to identify itself as gnome, their will still be problems for all the older verions out there (including FC1, FC2, RH9, RHEL3. Also as you say yourself sshing out to other or older OS/distro's will be broken too. So that only leaves local access and sshing from/to FC3 machines working, creating a nice FC island, if that iswhat we want I'll start using M$ again. We've had things working reasonable well for years, and as shown by my second patch, we can still have things working well. Without al the breakage described above, by just keeping things as they were. I've been a RH user since 3.x and I must say that I've never feeled less involved, then since RH has started this community oriented devel process. If you read al the comments here it is clear that most users want something which works in all scenarios not in 2 (and I say 1) out of three scenarios) Marking this bug as CLOSED. The new ncurses with the corrected home/end in the gnome terminfo entry corrects the problem described in the "summary" of the bug. o After talking with both the gnome-terminal/vte maintainer and the upstream ncurses maintainer - I am rejecting the notion of patching the "xterm" entry, which I am convinced is correct and should remain true to the source and something we should've done long ago. Fedora is the place to look forward, not hold on to broken behavior of the past. If you can get the patch to be accepted upstream (ncurses tarball), that is another matter... - I am open to the idea of patching non-"xterm" entries such as gnome, konsole, etc. Those patches should also be submitted upstream to ncurses if they affect all distros. o Arguments about gnome-terminal/konsole not working against the "xterm" definition belong in bug reports filed against gnome-terminal, vte, konsole, etc. o Arguments about vim not working belong in bug reports filed against vim. o Arguments about TERM not being set correctly /probably/ belong in bugs filed against component setup, as that's where the term.sh code will go. (either in profile or in bash... RHEL3 and FC2 gnome-terminal behave differently in these respects) o and finally, I invite arguments about feeling "less involved ... " with the "community oriented devel process" and about what "most users want" to transfer this argument to fedora-devel-list (despite comment 40). As for the arguments about "their will still be problems for all the older verions out there (including FC1,FC2, RH9, RHEL3)" which would "creating a nice FC island", I fail to see supporting users that refuse to update their boxes (with automated tools such as up2date/yum, or in the case of end-of-life RHL9, manually) as a problem worth solving, especially since we can control & push updates (such as the term.sh addition) to those distros. And furthermore, by keeping "xterm" pure, we're actually opening up the so-called "island"-- real xterm users on un*x boxen get a predictable and true xterm definition when they log into a FC/RHEL box. Also, I am skeptical about the claimed inconvenience of having to change terminal related settings (TERM etc) etc to make things work with older/alien boxen only applies to terminfo because of this reason: FC1 went full UTF-8 (RHL9 was UTF-8 for western euro only)-- which is a radical leap in functionality that breaks when connecting to/from 8-bit single byte distros (still the majority). In other words, users are still going to have to flip a whole lot of other things (the codeset in gnome-terminal, the locale settings within the session) from their FC (and RHEL4) defaults to "legacy modes" before they're going to get a decent text mode experience with older (and non-updated) boxes. FC has always been about minimalizing patches (especially kludges, which is what patching xterm to work with non-100% xterm compatible terms is) and looking towards the future (which is fixing vim, and fixing gnome-terminal-- and until they're /really/ xterm's and not just wanna-bes, you set TERM to identify them as wanna-bes) Havill, Thanks for the clear position, and for taking the time to answer. I wasn't aware that one of fedora's policies was to create an as much unpatched distro as possible. I'll try and take a look at your patches and create new bugzilla entries where appropiate using the components as described above. I know I just said I would create new bugzilla entries, but this so called fix, breaks a lott, so for convienence I'm entering things here, just lett me know if you want this in a new bug: 1) As warned for a couple of times, gnome-terminal uses the xterm terminfo to decide which esc-sequences to send for home and end. Since the new xterm terminfo entry has no entries for khome and kend, it falls backs to it defaults, which are ESC O H for home and ESC O F for end. The new ncurses-4.5-10 however, has khome and kend entries for gnome which contain ESC [ 1 ~ and ESC [ 4 ~. Thus the orignal problem stil isn't fixed. 2) The latest setup....rpm in rawhide doesn't contain term.sh 3) The new ncurses-4.5-10 has no entries for khome and kend, causing the problems as described in this bug with less, for xterm itself too. IMHO this is not acceptable! 4) Starting mc in a real xterm gives: *** err [lib/liblow.c(268)]: strncmp/isdigit/consolename failed 5) mc now things xterm is a dump terminal and asks for press any key to continue after every command executed 6) the ESC sequences send by F1-F4 by a real xterm don't match those in terminfo, this will brake apps! 7) if you do export TERM=xterm-xfree86 to fix 3-6, home and end don't match terminfo, since these are mapped to ESC [ 1 ~ instead of ESC [ H in the XTerm appdefaults file, this needs to be fixed by taking out all the keybindings in the appdefaults file, I don't know if you can yank out the scrollwheel bindings too, since I don't have a scrollmouse to test. 8) in the real xterm backspace is mapped to CTRL-H in terminfo, but is mapped to asc-del (\177) or CTRL-? in the XTerm appdefault file by the: "*VT100*backarrowKey: false" line. 9) if you fix 8 by removing the trouble making line from the XTerm app-defaults, Emacs users will become unhappy since pressing backspace gives them their helpscreen. 1-2 should be easy to fix 3-6 are there because xterm in terminfo now uses xterm-r6, this could be fixed by making xterm teerminfo use xterm-xfree86. But afaik you don't want that. Another fix is to make xterm report itself as xterm-xfree86, by putting a: "*termName: xterm-xfree86" line in the XTerm appdefaults. 7-8 can be fixed by removing most of the modifications to the XTerm appdefaults. 9 is an emacs bug, not a ncurses or terminal problem. Well thats what a quick scan of gnome-terminal and xterm has found me sofar. I'll try konsole next, do some more testing and report it all here. Additonal note, it might be a good idea to lett term.sh only muck with TERM if TERM equals xterm. Created attachment 101993 [details] new term.sh incorperating suggestion made in comment 60 A quick scan of konsole has found me the following problems: -no terminfo entry for backspace, home and end -the ESC sequences for F1-F5 are wrong Created attachment 101995 [details] patch fixing problem1 in comment 59 and all problems in comment 62 I've got todo some other stuff now I'll write a patch for the XTerm appdefaults fixing 3-8 in comment 59 tomorrow. Some more thougts: -all the changes we do to terminfo need to be mirrored to termcap. Actually I believe termcap as such should not be a package, and /etc/termcap should be a part of ncurses so that they can't get out of sync. p.s. Since there are still some problems with this bug, and since the original bug still is there, I believe this bug should be reopened. I have a to-do item to update (and fill in missing information) for the "gnome" terminfo entry, in ncurses. Once that's done, there should be no reason why gnome-terminal cannot use that. Thomas, Following the path we now have choicen I fully agree, my patch is just a quick fix for the most obvious problems for now. Beware though of the fact that gnome-terminal determines how to behave partly on the xterm terminfo entry, so what should be in the gnome terminfo entry depends on what the xterm terminfo entry contains. The differences should be smaller now - see ftp://invisible-island.net/ncurses/terminfo.src.gz Thomas, I took a quick look remarks sofar: -gnome terminal can send F1 and F10, you can unbind them from the menu somewhere in the options / preferences. If you do this they work just as xterm's -konsole by default uses the konsole-xf4x keyb layout, not konsole-linux -konsole sends the same ESC-sequences for shifted F-keys as xterm does, and I've submitted a patch upstream to also make it send the xterm ESC-sequences for the ctrl and alt modifiers which it currently ignores. I would like to see konsole-xf4x using xterm-pc-fkeys, this will work for the shift modifiers right now and also prepare ncurses for the patch I've submitted (which hopefully will get applied) Created attachment 102012 [details] patch taking terminfo.src to the version given by Thomas Dickey in comment 67 Created attachment 102013 [details] patch fixing the problems mentioned in comment 68, this one should be applied on top of the previous one For konsole this patch also maps kbs to \177 instead of ^H, since that is what the current konsole does. Testing konsole on Debian yesterday, I did not see it respond to any modifiers other than shift. I made a to-do note to make a reduced description for it, but xterm-pc-fkeys does not apply to the version of konsole I was testing. (I wasn't sure if konsole-xf4x was the default, or whether Debian had customized it - likewise the ^H/^? choice - intended looking at the upstream source to get a definitive answer). It would be nice if gnome-terminal and konsole's changelogs reflected the places where they copy functionality from xterm (so far I've found the changelogs and documentation to be useless). I can't find any patches changing konsole's behaviour in the kdebase src.rpm so I assume that kbd=\177 and the xf4x layout are the upstream defaults, since thats the behaviour I have on my FC3 test system. You're right that currently konsole only uses the shift modifier, I've modified my patch to reflect this, since I'm not sure that my konsole patches are going to be excepted upstream, and because my patches don't support combined modifiers and thus need rewriting. Created attachment 102014 [details] patch fixing the problems mentioned in comment 68 updated to reflect the problems with this patch mentioned in comment 71, this one should be applied on top of the terminfo.src update Created attachment 102015 [details] improved patch fixing the problems mentioned in comment 68 updated to reflect the problems with this patch mentioned in comment 71, this one should be applied on top of the terminfo.src update My last patch (comment 73) redefined F10 - F12, without this being nescesarry. This version cleanes this up. Checking the upstream source, it appears that only the vt420pc flavor of konsole uses ^H. Also, it appears that the default (upstream) keyboard setting is the XFree86 4.x.x (for some time - so it's likely I was seeing a local customization before). Hmm, Using TERM=xterm-xfree86 and a fixed appdefaults to fix problems 3-8 in comment 59 causes the following problem: When opening a textfile in joe the first character of some lines is missing. Joe does know that it should be there, pressing ends moves the cursor one position past the last char of the line, but for some reason the first char isn't displayed and the rest of the line is shown shifted one char to the left. Using TERM=xterm-xf86-v43 makes this problem go away. Thomas do you have any thoughts on this? Are you using gnome-terminal? Wrapping behavior is one of the
places where gnome-terminal doesn't match (anything except
konsole).
Only last week I got mail from someone asking me to bless it.
There's a demo script in
ftp://invisible-island.net/temp/my_eol
which may show the same sort of thing.
Another thought - I've noticed in the past week or so that my
shells are omitting (in some cases) the first character of the
input from the display. Other than seeing that if I change $TERM
to xterm-xf86-v44 that fixes it, I haven't pinpointed the problem.
It could either be another symptom of the workaround I've made for
tgetstr("me") versus Debian #254316, or another issue undetermined.
If it's the same as #254316, the problem will go away with the
next update for ncurses. (I'm normally running screen, which would
be affected by the same problem, but recompiled that w/o otherwise
modifying my shared ncurses libraries so I could look for other
impact of that bug).
Correcting, Thomas, I think you meant ftp://invisible-island.net/temp/my-eol because that is the onliest similar reachable file ;-) yes. Anyway, there are two that come to mind, depending how I read your comment. Thomas, The first char missing problem is with the real xterm. That's why I'm using TERM=xterm-new, to fix all the problems you get with the real xterm when using TERM=xterm (which is the same as TERM=xterm-r6) It sounds to me like it the same problem you're having with the first char missing on the cli. I just tried using TERM=xterm-xf86-v44 and that fixes it (so does TERM=xterm-xf86-v43) Could you give a pointer to this new ncurses which you are talking about, then I can see if that indeed fixes it. Maybe FC3 should use the terminfo.src shipped with ncurses, with just a few minimal patches applied, since your new version seems to break other stuff. That's only in source - my rollup patch at ftp://invisible-island.net/ncurses/5.4/patch-5.4-20040711.sh for instance. The particular file that changed happens to be one that hadn't changed recently: ncurses/tinfo/lib_termcap.c so it would be possible to update just that file. To make it simple, see ftp://invisible-island.net/temp/lib_termcap.zip The difference that matters in the terminfo are the rmacs, smacs, enacs, sgr and sgr0 strings. The lib_termcap.c code filters the sgr0 string, as I noted in the Debian bug report. I just installed up-to-date libraries over Debian's, to check if that fixes things as well - they're upward-compatible as far as I know. Thomas and Havill, I've tried Thomas' ncurses patch, upgraded with to the newer terminfo.src he has pointed out earlier with my konsole and gnome patches applied. This does indeed fix the first char on a line missing problem, but this has problems with lines which are longer then 1 line and thus need to be wrapped, this goes "funky" both on the bash prompt and in joe. So I have abandoned the new terminfo.src, and instead just took the gnome and konsole entry changes from the new terminfo.src, and fixed the problems with these described in comment 68 and 70. If I've understood Thomas correct he agrees with these fixes. I'll attach a patch for Thomas' gnome and konsole updates with my fixes integrated. I've done some more testing and I can't find any regressions with the plain ncurses-5.4 terminfo.src with just this patch applied, using term.sh and making the nescesarry changes to xterm app-defaults. This goes for xterm, gnome-term and konsole. May I suggest applying my (latest) patch instead of the current broken home_end patch and closing this bug once and for all. Created attachment 102025 [details]
tarball containing dirs with patches to either fix plain ncurses-5.4, or to take it to the latest ncurses-snapshot and terminfo.src and then fix it
I made a booboo (once again) The long line problems are with the latest terminfo xterm-new entry, but I was still using a plain (old) FC2 xterm. These problems go away by upgrading xterm to the FC3-test1 release. Still I think just applying a minimal set of fixes to plain ncurses-5.4 is the best approach, since the snapshot version are just that: snapshots. Created bug 128138 for the needed xterm appdefaults changes odd - I don't recall any changes to xterm itself that would affect line-wrapping. I'll have to find more about that. I seem to recall that fc2 xterm is patch #179 - is that right? A couple of typescript files (from 'script') running the good and bad cases for the long bash prompt might show me what's actually happening. Forget I ever said that, FC2 indeed ships with xterm-179 and I just downgraded to this and I can't reproduce the problem. Maybe I had TERM set to something funky I have been experimenting so much with this stuff that I appearently did something weird. Right now I can run the latest ncurses snapshot with the latest terminfo.src applied over it with my fixes to gnome-terminal and konsole on top of it and it works just fine with xterm-179 and xterm-192. good (I'm of course interested if there's a problem that we can reproduce, but don't recall any new problems with the current ncurses snapshot - just a number of unfinished details) Created attachment 102035 [details] slightly tweaked term.sh Same as the term.sh proposed with comment 60, except surround $TERM in double quotes so that an unset TERM doesn't cause the script to fail with a syntax error. I felt that one ("" around $TERM) coming about two minutes after I had
attached my term.sh :)
One last note, if you're going to use the ncurses snapshot and/or the
updated terminfo.src: I just noticed that with the updated
terminfo.src from Thomas xterm now longer uses (points to) xterm-r6
but points to xterm-new.
Afaik you wanted xterm to point to xterm-r6, as it does with a vanilla
ncurses-5.4 terminfo.src. So if you're going to use the updated
terminfo.src don't forget to patch it to make xterm point to xterm-r6.
That's a configure option (--without-xterm-new). btw, I'm not sure that term.sh will work properly with rxvt. On my system for example, rxvt sets $COLORTERM to "rxvt-xpm". I'm seeing the same over here, suggested fix: -change the if test $COLORTERM=gnome-terminal test to a case, then do the current gnome thing for gnome, do nothing for rxvt* and do the konsole thing for * -even better would be to patch konsole to properly set COLORTERM, and then only do the konsole thing when COLORTERM=konsole p.s. When is this bugged going to gert really fixed? My patches taking just the changes for konsole and gnome-terminal from the latest bleeding edge ncurses/terminfo and intergrating these into terminfo from a clean ncurses-5.4 are pretty straigtforward and safe IMHO. And currently things are still broken in rawhide. (I thought I saw a comment from Havill that he's on vaction til August 1). no i'm still here. looking to push tonight after i test the rxvt and COLORTERM 5.4-11 looks very good, BUT currently TERM=xterm in terminfo is the same as TERM=xterm-new, afaik you wanted that to be the same as TERM=xterm-r6. In order to get TERM=xterm to be the same as TERM=xterm-r6, add --without-xterm-new to the %configure flags. 5.4-11 works for only partially for me, but that isn't still enough...if I do "less <rpm-file>" for example, home/end works. But if I do "rpm2cpio <rpm-file> | less", there home/end doesn't work. This already worked with 5.4-7 for me... FYI: I did this (reproducable) over a ssh connection and also at the konsole directly (ttyX). Seems so, that a further fix is needed to solve this problem also. Ehm, your piping a binary file through less, who would want to do that? Hans, I didn't test piping text files through less yet, but I think when we try to solve this issue, we should do it completely and in the right way, otherwise we can fallback to the old non-working stuff... ;-) Binary data streams can contain anything, and thus also control-esc-sequences which completly mess up the terminal. Trying to display binary in a terminal us just ehm stupid. Well, I can't completely agree with you anyway, so THIS already worked with previous versions - why shouldn't it possible to make it here also working?! A quick check of the output from less in the two cases shows me that when you pipe to it, it simply cat's the file's output until it gets something interesting (like deciding that it's reached the end of the screen). In contrast, running from a file, it does some additional filtering). If you really need to cat binary files to the screen, you should file a bug report against less, since it is not doing what you want. I would like to add a few comments for konsole users:
1. Files /etc/DIR_COLORS and /etc/DIR_COLORS.xterm need to have
a line "TERM konsole".
2. /etc/bashrc (setup-) that sets the window title is based on "xterm"
(fancy titles), one needs to modify this to do the same for konsole.
Of course if a remote system checks for xterm to set this and you
come in as konsole.... you are out of luck!
And /etc/DIR_COLORS*... why isn't ls actually just using the termcap/terminfo? You do get some color without the /etc/DIR_COLORS* but some things like files with .rpm extension etc. do not seem to get color while directories etc do appear in color. I don't know why. I'm going to remove the term.sh script for now after testing against both current OSS apps (the term detect code for the xterm title bar in bashrc for example) and proprietary apps. Too many apps check for "xterm" and don't check for anything else. It's too bad terminal types are similar to XML MIME types, were you can tell if a certain format is XML based-- code is usually written to assume that if the TERM isn't "xterm", then it must be completely 100% non-xterm compatible. The current gnome-terminal seems to work with both gnome and xterm-- so long as the terminfo/termcap is not generated with --without-xterm-new-- as home/end seem to fail to work against the newest gnome-terminals. Well, it would be nice if "ls" simply used termcap/terminfo to decide if the terminal supports color (and got rid of the constants). Adrian's last comment confuses me: as I understand it, gnome-terminal doesn't recognize the escape sequences used in xterm-new for rin/indn. Wait a minute, So now it has been decided that term.sh is going to be dropped, and I was just starting to like the idea :| But if you're going todo this then you will need to: -modify terminfo.src so that the xterm entry: *explicitly tells not to habe rin/indn, since this breaks for example scrolling longer mails in mutt in gnome-terminal *probably best to make kbs be \177, instead of the xterm-new ^H, since \177 is what both konsole and gnome-terminal send *uses xterm-new *maybe Thomas is willing to make an xterm-compat terminfo entry for this?? -modify the real xterm to send \177 for backspace (there is a resource setting for this) -investigate the mutt line drawing problem you have mentioned in this bug About my last comment: maybe I was a bit quick to suggest that terminfo should be modified to reflect the fact that kbs sends \177 instead of ^H On second thought it might be better to fix gnome-term and konsole here to match xterm. Since xterm is the one and only real xterm. We do still need to suppress the rin and indn capabilities though. Here's a datapoint that might be of interest. Although this only happens with screen (and in fact only fc2 screen, fc1's seems fine) it sounds like the same bug. Basically solaris9 gnome-terminal (2.0.1), ssh to fc2 box, start up screen (TERM=xterm), use tab command completion in bash -> text gets messed up like this: it-gw13 (pp):/var/tmp >ls nadvsh sparse-bk srss_3.0 srss_linux_3.0.tar it-gw13 (pp):/var/tmp >ls srss_<3 x TAB> and it changes to it-gw13 (pp):/var/tmp >ls slinux_3.0.tar TERM=xterm-xfree86 before screen -x and it's fine. That appears to be an unrelated bug: rin/indn relate to indexing (scrolling), and the example given would not use that feature. *** Bug 105781 has been marked as a duplicate of this bug. *** With a current Fedora Development system neither "End" nor "Pos1" is working - tested over ssh and terminal directly. Created attachment 103352 [details] ncurses-5.4-color_xterm.patch *grmpf* with the latest ncurses-5.4-13 from Development I've got the same problem further on as already for an older ncurses version described before in comment #115. For my personal use, I merged Hans de Goede's original patch from comment #5, which still works for me (maybe because I don't use GNOME?! *smile*). I attached it here, maybe someone wants to use it and don't want to re-merge it ;-) I'm still curious whether we get this problem solved - more or less - elegant some day ;-) Since it is bug squash week, can we please finally fix this one? *** Bug 132817 has been marked as a duplicate of this bug. *** *** Bug 134300 has been marked as a duplicate of this bug. *** I submitted Bug 134300, and when it got marked as a duplicate of this bug, I came here. After reading everything up till now, I have to admit that this is a big hairy problem. To summarize my bug: Fresh install of FC3-test2, graphical log in to gnome, start gnome-terminal from the menu, start lynx to an ftp site, navigate around a bit, get a mangled display. vim similar problem. ^L fixes. Initially, I thought "Hey, why doesn't gnome-terminal just say TERM=gnome-terminal?" Problem: no such terminfo/termcap entry. Ok, how about gnome-terminal setting TERM=gnome? Problem: no such terminfo/termcap entry on other systems, as pointed out by notting. My solution: A similar solution to what was posted here: a term.sh hack that says if COLORTERM=gnome-terminal and TERM=xterm, set TERM=vt100. I use vt100 because it makes the problems go away, it's available on other unixes, and I'm pretty confident that it hasn't been touched by anybody trying to "improve" things. Note: I don't use function keys or any cursor keys beyond the four arrows, which may explain why I'm fine with vt100. Oh, and offtopic: the backspace key has always been a problem... ^H or ^? ? Fortunately, tcsh takes both. Unfortunately, when I looked last, it didn't set stty, so programs could still get it wrong. I think I've got it fixed in my startup scripts, but *man* that's a problem that wouldn't be missed. Thanks, and keep up the good work! Is it possible, that we solve this issue up to Fedora Core 3 in a fine way?! Today, I had to work with a SuSE Linux 9.1 and I noticed, that they have the same problem... ;-) Sorry for stepping in late in the discussion. I hope my feedback will be someway useful. I'm using a fully updated FC2 remotely, without even X11 installed. I connect to it remotely via SSH and I get TERM=xterm from a RedHat9. My home/end keys doesn't work in mutt and setting TERM=linux fixes the problem. re-tested with latest rawhide FC3 and confirmed working pgup/pgdown/home/end in less on xterm/console/gnome-terminal/konsole Does this fix the additional problem about display corruption in 'lynx' and 'vim' with TERM=xterm and gnome-terminal? If not, then please undelete Bug 134300. Thanks! I've been doing some intensive testing /research related to this bug, the results: -ncurses/terminfo now seems 100% ok. -gnome-terminal has some inconcistencies with terminfo and the real xterm reported upstream: http://bugzilla.gnome.org/show_bug.cgi?id=157448 -konsole has some inconcistencies with terminfo and the real xterm reported upstream and to RH: http://bugs.kde.org/show_bug.cgi?id=92749 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=138191 -xterm should not identify itself as xterm-new see my latest comment: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=128138 *** Bug 121922 has been marked as a duplicate of this bug. *** |