(10:01:59 PM) racke: ding dong, who is present for IRC meeting? (10:02:21 PM) __justin__: i am i am (10:02:55 PM) racke: agenda: http://www.icdevgroup.org/pipermail/interchange-users/2009-November/051412.html (10:03:01 PM) racke: any additions? (10:03:22 PM) ***Guest007 is present (10:03:46 PM) ***Guest007 has no additions (10:05:45 PM) k2b3: here for a while, but in body only! (10:05:52 PM) racke: haha (10:06:05 PM) racke: I want to address also: http://www.icdevgroup.org/pipermail/interchange-users/2009-November/051402.html (10:06:07 PM) ***endpoint_david here (10:07:01 PM) racke: I wonder if there is more code affected from [data session arg] XSS exploits (10:08:43 PM) endpoint_david: that one was due to issues with outputting the unescaped data in a server-generated message, right? (10:09:29 PM) racke: yes, because [data session arg] is basically CGI input (10:09:40 PM) racke: let me put together a list from a grep (10:10:35 PM) endpoint_david: I think escaping in-place is the wrong answer, should be done at the site of output (10:10:44 PM) endpoint_david: (if I remember the discussion at the time) (10:10:55 PM) endpoint_david: *sites (10:11:21 PM) racke: so you support doing it at on page level? (10:15:40 PM) endpoint_david: I think that's better than munging the data (10:15:54 PM) endpoint_david: in the cases where there's an actual page (10:16:11 PM) racke: but no user will do the escaping (10:16:14 PM) endpoint_david: otherwise, you might be doing calculations/lookups on the escaped value, which doesn't seem like the right thing (10:16:30 PM) endpoint_david: I may be misunderstanding the situation (10:16:53 PM) endpoint_david: what do you mean by "no user will do the escaping?" (10:17:36 PM) racke: I think it is really unlikely that they upgrade/fix their affected pages (10:17:56 PM) racke: it is even unlikely that they upgrade Interchange at all, but that will eventually happen (10:18:07 PM) endpoint_david: ok, I see the issue you're talking about (10:18:32 PM) racke: and did you see the grep? no calculation (10:20:16 PM) endpoint_david: arguably this shouldn't need anything escaped, but take this as an example: (10:20:16 PM) endpoint_david: [if type=data term="products::sku::[data session arg]"] (10:20:20 PM) endpoint_david: from forum_display.html (10:21:30 PM) endpoint_david: would anything break if there was something that was now escaped? (10:21:32 PM) racke: what is your argument? (10:21:54 PM) racke: only mainframe SKUs for Jon O:-) (10:23:17 PM) endpoint_david: well, what all is being escapde, just < and > ? (10:24:13 PM) racke: we don't allow scripting characters in the page name, so why should we allow them in mv_arg? (10:24:37 PM) racke: Page.pm, line 59 (10:25:17 PM) endpoint_david: ok, what about userdb names? (10:25:31 PM) endpoint_david: dist/standard/pages/member/delete_addresses.html: [userdb function=delete_shipping nickname="[data session arg]"] (10:25:49 PM) racke: scripting characters in nicknames? would be insane (10:29:49 PM) endpoint_david: yeah, they look fine to me (10:31:09 PM) Guest007: but in put such as '< 5' could be normal input .. right? and you'd want to store it 'as is' ... or not? (10:31:48 PM) racke: I don't see how '< 5' can appear as normal input for $Session->{arg} (10:32:15 PM) endpoint_david: where is $Session->{arg} populated, from $CGI->{mv_arg}, right? (10:32:25 PM) racke: yes, just as is :-) (10:33:30 PM) endpoint_david: my concern isn't necessarily with the pages in standard, those look fine, but in custom pages where people are using the [data session arg] usertag/$Session->{arg} themselves (10:33:56 PM) batrams [i=600bee22@gateway/web/freenode/x-bntdcxixdaxbhmyt] entered the room. (10:34:02 PM) racke: hello batrams (10:34:07 PM) Guest007: I'd say it is ones responsibility that if you output data session arg to escape things (10:34:13 PM) batrams: Hi! (10:34:21 PM) Guest007: that it has not been done in standard is sloppy (10:34:28 PM) __justin__: this is me as an outsider, but i would expect anything that is in session [data session] to already be cleansed (10:34:37 PM) Guest007: and perhaps should be fixed in a couple more locations (10:34:42 PM) racke: I can only second __justin__ (10:34:44 PM) hexfusion: is lurking (10:34:51 PM) racke: hello hexfusion (10:34:57 PM) hexfusion: :) (10:34:58 PM) endpoint_david: Guest007: that's my general feeling as well; I just don't like to munge data in general that may be used elsewhere (10:35:11 PM) endpoint_david: particularly if it's a behavior change (10:35:19 PM) Guest007 is now known as thunder (10:35:47 PM) racke: my general feeling is that security isn't treated seriously in the ICDEVGROUP (10:35:53 PM) endpoint_david: now if we wanted to change the semantics of the [data] usertag so it escaped its output by default, I could get behind it (10:36:13 PM) racke: [data] in general does this already (10:36:33 PM) endpoint_david: ok, what parts don't, [data session ...]? (10:36:46 PM) endpoint_david: that'd be what to fix IMHO (10:37:07 PM) ***endpoint_david resorts to the source (10:37:45 PM) racke: hm I thought so (10:37:54 PM) racke: but it does only (10:37:54 PM) thunder: change it as you like it, won't make me sleep less ... I just do not see why input would have to be cleansed ... it is output that has to be cleansed (10:37:57 PM) racke: sub ed { (10:37:57 PM) racke: return $_[0] if ! $_[0] or $Safe_data or $::Pragma->{safe_data}; (10:37:57 PM) racke: $_[0] =~ s/\[/[/g; (10:37:57 PM) racke: return $_[0]; (10:37:57 PM) racke: } (10:38:24 PM) racke: thunder: because of the GIGO effect (10:38:28 PM) thunder: well add a < and > there and you're safe enough I suppose (10:38:35 PM) racke: Garbage in = Garbage out (10:38:55 PM) thunder: but what is your Garbage could be my Treasure ... (10:38:59 PM) thunder: how do you know ... (10:39:36 PM) __justin__: i agree with thunder, it should be cleaned prior to filling the session (10:40:02 PM) thunder: but if you want to be safe for those who can't be for themselves and with that bother those who know how to be safe and who will have to work around unwanted safeguarding then add some '<' and '>' to that sub ed and you are "safer" .. (10:40:12 PM) racke: we are programmers not garbage workers so we should treat us as such :-) (10:40:27 PM) hexfusion: ha! (10:40:57 PM) endpoint_david: hmm, I'm sure they are sites I've worked on where adding < and > to the data munging will break existing places where html is stored/put out (10:42:11 PM) endpoint_david: (to [data] in general now, not just [data session arg] (10:42:12 PM) endpoint_david: ) (10:42:49 PM) racke: I suggest to make this configurable, so people can choose their level of protection, e.g. using an regex or some kind of filter (10:42:52 PM) endpoint_david: i.e., how do you distinguish between "we legitimately want to output our pre-sanitized html" vs "I want literal html to show up here" (10:43:32 PM) thunder: aparently there is a 'filter=encode_entities' to add to [data base=session ... so that level of protection is already there .. you only should be using it when you need it ... (10:44:06 PM) thunder: again I have no problem if you want to solve it in a different way .. make it configuration item 5,302,012 which we will not remember unless it is switched on by default (10:45:00 PM) thunder: So I suggest that racke will suggest in a post to the ml that it is made configurable and have people comment on that ? (10:45:08 PM) thunder: So we can move on to the next topic? (10:45:28 PM) racke: hehe (10:45:44 PM) racke: thunder likes quick meetings :-) (10:45:46 PM) thunder: (or put in a patch and after that post to the ml, bigger chance that it stays in ;)) (10:46:16 PM) thunder: nope thunder does maths and 45 minutes per topic will mean that I won't be sleeping at all tonight (10:46:29 PM) racke: ok ok (10:46:41 PM) thunder: #304: Encode mime_name method missing in Encode < 2.21 (10:46:55 PM) racke: no let's do another one before (10:47:00 PM) thunder: there is a good patch available from peter ... I think that patch solves it (10:47:14 PM) thunder: so someone patch the code and we can move on to the next topic (10:47:26 PM) thunder: 1.5 minutes for that point, not bad aye? (10:47:31 PM) racke: #331: Embedded Perl Encode::Alias problem (10:47:37 PM) racke: this is much more nasty (10:47:58 PM) ***thunder will reiterate my point of #304 after #331 then (10:48:12 PM) racke: I even thought about putting a copy of Encode::Alias inside Interchange with the 'require Encode' removed (10:48:28 PM) racke: this fixes the bug (10:48:58 PM) thunder: Sounds like the best solution if you want to keep the schedule for the release (10:49:03 PM) ***thunder +1's it (10:49:43 PM) thunder: Someone would have to keep up to speed with Encode::Alias to grab newer versions and cut out the one liner ... but that seems to be acceptible bother to fix this nasty issue .. (10:50:34 PM) racke: in addition we could open a bug report on perl (10:51:21 PM) ***thunder won't be making that bug report, but fine if anybody else feels like it ... (10:51:32 PM) thunder: any objections to racke's idea? (10:51:35 PM) thunder: going once .... (10:51:39 PM) thunder: going twice .... (10:51:50 PM) thunder: soooooooooooooooold! (10:51:56 PM) racke: haha (10:52:15 PM) racke: moving, moving, moving :-D (10:52:27 PM) racke: #304: yeah, let's just apply pj's patch (10:52:29 PM) thunder: #304? :) (10:52:34 PM) ***thunder grins (10:52:37 PM) thunder: good (10:52:43 PM) thunder: #255 (10:54:35 PM) endpoint_david: I'd run into some issues with just commenting out the 'require Encode' (10:54:40 PM) endpoint_david: but I don't rightly recall them (10:55:10 PM) endpoint_david: I have an alternate patch for #304, but don't care which one goes in (10:55:12 PM) thunder: too late endpoint_david , you cannot run into issues :) (10:55:28 PM) racke: please materialize your patch, David (10:55:47 PM) racke: which issues? (10:56:02 PM) racke: issues with Interchange or with other Perl scripts? (10:56:20 PM) k2b3 left the room. (10:56:41 PM) racke: isn't require Encode in Encode::Alias a no-op inside Interchange? (10:57:16 PM) endpoint_david: racke: interchange, I believe, which was the only place I was testing it (10:58:10 PM) racke: I still favour to try it out (10:58:34 PM) racke: we can back it out or further mangle it if you can name your issues (10:59:21 PM) endpoint_david: http://www.paste.me.uk/1360.html (11:00:15 PM) endpoint_david: note I didn't try it with a separate Encode::Alias in the IC tree, jsut modifiying the system Encode::Alias (11:00:38 PM) thunder: nice path endpoint_david (11:00:40 PM) thunder: atch (11:00:43 PM) racke: yeah I did the same, I had no issues yet (11:00:44 PM) thunder: patch (11:00:48 PM) thunder: pfffff.... (11:00:51 PM) endpoint_david: that seems like a recipe for disaster, though, as I'd expect interdependencies between Encode and Encode::Alias (11:01:22 PM) racke: David, would you mind to apply this patch? (11:01:47 PM) racke: no the question is what does the require Encode do if we have it already loaded? (11:02:07 PM) endpoint_david: racke: the interactions are mainly in Safe, not in the general code (11:02:16 PM) endpoint_david: sure, I can apply that patch (11:02:28 PM) endpoint_david: I have to runa quick errand, back in 10-15 mins (11:02:48 PM) endpoint_david: racke: what version of Perl do you run locally (11:02:52 PM) endpoint_david: ? (11:03:06 PM) racke: 5.10.1 (11:03:26 PM) racke: Squeeze :-) (11:04:57 PM) racke: #255: Encoding of mail headers (11:05:23 PM) racke: David and I discussed this last week, but lacked the conclusion (11:05:30 PM) thunder: ok (11:05:48 PM) thunder: which piece of code in Interchange is now actually reponsible for sending mail? (11:06:00 PM) GodFather [n=rcc@12.30.25.130] entered the room. (11:06:11 PM) thunder: I think there is something in Util.pm, Email.pm and Interpolate.pm (11:06:14 PM) racke: hello GodFather (11:06:21 PM) racke: we are looked at send_mail (11:06:36 PM) racke: but if we can patch it there, we could do it elsewhere do (11:06:39 PM) racke: sorry too (11:06:43 PM) GodFather: racke: what's up! (11:07:07 PM) racke: GodFather: http://www.icdevgroup.org/pipermail/interchange-users/2009-November/051412.html (11:08:31 PM) GodFather left the room (quit: Client Quit). (11:08:42 PM) thunder: that was too much too handle hehe (11:09:06 PM) racke: he just knows that his sheep are busy with the right things to do :-) (11:10:38 PM) thunder: ########################################################################### (11:10:38 PM) thunder: # Old functions, preserved more or less as-is. To be called when no (11:10:38 PM) thunder: # useful compatibility wrapper can be made. (11:10:38 PM) thunder: # (11:10:38 PM) thunder: # Vend::Util::send_mail (11:10:58 PM) thunder: hehe (11:11:10 PM) thunder: sub send_mail_legacy { (11:11:16 PM) ***thunder likes to handle the cut without paste keys :) (11:11:35 PM) racke: Email.pm is abandoned right now (11:11:58 PM) thunder: ok so it is 'sub send_mail' in Util.pm which is the one? (11:12:21 PM) thunder: or Email.pm should be unabandoned first? ... I remember vaguely that it was something davor was working on a couple of years ago? (11:12:54 PM) racke: there is more than one :-) (11:13:56 PM) racke: [email-raw] doesn't call sendmail (11:14:34 PM) ***endpoint_david back (11:14:41 PM) ***thunder thought there was something with 'raw' for e-mail sending (11:14:45 PM) matjones left the room (quit: "ChatZilla 0.9.85 [Firefox 3.5.5/20091102152451]"). (11:14:51 PM) endpoint_david: thunder: you're thinking the email-raw.tag (11:15:16 PM) racke: [email-raw] doesn't call send_mail function (11:16:06 PM) ***endpoint_david can read, honest (11:16:33 PM) racke: sendmail could mean the sendmail program too :-) (11:16:47 PM) thunder: $ok = send_mail(\@head,$body); (11:16:54 PM) thunder: that is in email_raw.tag (11:17:00 PM) thunder: so it does call it in some occasions (11:17:50 PM) racke: yeah but you have to aware that it doesn't go through all the times (11:18:25 PM) thunder: exactly (11:20:04 PM) racke: and the last line: (11:20:07 PM) racke: return $opt->{hide} ? '' : $ok; (11:20:13 PM) racke: should be just (11:20:16 PM) racke: return $ok (11:20:18 PM) thunder: and you can't just start to MIME stuff in email_raw because if it comes though the one going to send_mail it would double encode (11:20:20 PM) racke: ok ? (11:20:39 PM) thunder: why is that ? (11:20:49 PM) thunder: if you hide is on it will return nothing (11:21:12 PM) racke: the hide attribute is automatically recognized now I think (11:21:18 PM) thunder: else it could return '1' (11:21:30 PM) thunder: ah ok (11:21:31 PM) endpoint_david: pushed the mime_name patch (11:22:57 PM) racke: if ($attr->{'hide'}) { (11:22:57 PM) racke: $routine->(@args,$tmpbuf); (11:22:57 PM) racke: } (11:22:57 PM) racke: elsif($attr->{reparse} ) { (11:22:57 PM) racke: $$buf = ($routine->(@args,$tmpbuf)) . $$buf; (11:22:57 PM) racke: } (11:22:57 PM) racke: else { (11:22:57 PM) racke: ${$self->{OUT}} .= $routine->(@args,$tmpbuf); (11:22:57 PM) racke: } (11:23:00 PM) thunder: but the filter for the header should be applied in the 'else'in email_raw and in send_mail in Util.pm ... right? (11:23:35 PM) racke: yes, I guess (11:24:25 PM) thunder: but was Email.pm now the one that was to be used in the future? Or is Email.pm a filethat should be removed from the source and then to fixeverything else thatis broken? (11:25:14 PM) racke: well it would make sense to consolidate all mail functions in one module (11:25:31 PM) racke: but for now we should fix this issue (11:27:21 PM) thunder: so the lines in the elsif in email_raw with regard to creating the @head should probably be also used for the 'else' loop where in the else you canthen loop through the @head and MIME those (11:27:30 PM) endpoint_david: racke: what is the specific issue, the encoding of the headers? (11:27:30 PM) endpoint_david: +1 for consolidating APIS (11:27:45 PM) racke: yes (11:27:55 PM) thunder: and then print VEnd::MAIL $encodedheaderlines and then print Vend::MAIL $body ... (11:29:11 PM) thunder: and it might be good to go back in time to find some posts about email with regard of what work was done , how and why ... to determine if Email.pm could be tossed (11:30:51 PM) racke: meeting log of last week is now here: http://ftp.icdevgroup.org/interchange/meeting/irc_meeting_20091117.txt (11:36:10 PM) thunder: so the problem is not just having the MIME encoding working well but the overall encoding of the texts such as Subjec .... (11:36:20 PM) racke: we tried a number of variants, but none really worked for me, often I ended up with double encoding (11:36:59 PM) thunder: I'm too tired to think of the right solution for this one .. I have been messing with this in another application sometime as well ... especially when it came to longer lines than 72 characters or something (11:38:01 PM) thunder: and if it worked for the one mail program it was broken for the next ... totally annoying (11:38:03 PM) bitsfritz [n=bitsfrit@p57A53298.dip.t-dialin.net] entered the room. (11:38:16 PM) racke: hello bitsfritz (11:38:43 PM) racke: for F-Shop I'm using MIME::EncWords::encode_mimewords which works for the header encoding (11:39:12 PM) bitsfritz: hello racke (11:39:37 PM) thunder: ok ... for whom/what does it not work then? (11:39:49 PM) racke: with the current source code it is broken (11:39:58 PM) thunder: at least that is what I got from the notes of the previous meeting ... (11:40:03 PM) racke: because it doesn't encode mail headers with UTF-8 (11:40:31 PM) racke: mail clients may still happily display latin-1 (11:40:39 PM) thunder: ok (11:40:51 PM) racke: but encoding is a must (11:41:15 PM) thunder: yes it is for anything other than latin- (11:41:19 PM) thunder: 1 (11:41:39 PM) thunder: 5 minutes before I'm off (11:41:40 PM) racke: did you try it with Greek :-) (11:42:12 PM) thunder: the mails I'm sending out are screwed up ... is solved when changing view->encoding-> utf8 ... (11:42:15 PM) racke: ok let's adjourn the meeting until next week (11:42:24 PM) racke: try (11:42:26 PM) thunder: but they are surely not sending out correctly (11:42:33 PM) thunder: actually I am currently having problems with 5.7.1 (11:42:38 PM) racke: CodeDef mime_encode_words Filter (11:42:38 PM) racke: CodeDef mime_encode_words Description Find MIME type of filename (11:42:38 PM) racke: CodeDef mime_encode_words Routine < $Tag->filter({op => 'mime_encode_words', (11:43:16 PM) racke: body => 'Bestellbestätigung F-Shop'}), (11:43:30 PM) batrams left the room (quit: "Page closed"). (11:43:39 PM) racke: yeah, the description is bogus, I'm sorry (11:43:54 PM) thunder: I will have a go with the mime_encode_words sometime yes ... (11:44:04 PM) thunder: I still need to migrate to the new server (11:44:21 PM) thunder: One more point to address I wanted today and tht is opening RT (11:44:31 PM) racke: ah yes (11:44:33 PM) thunder: I gave it a +1 .. haven't seen anybody else respond to that (11:44:40 PM) racke: ok you won! (11:44:47 PM) thunder: So wanted to mention that one for the record (11:44:56 PM) racke: ok, duly noted! (11:45:15 PM) racke: thanks for your assistance (11:45:21 PM) thunder: ok .. well I am going to head to bed ... laterz! (11:45:28 PM) racke: me too (11:45:33 PM) racke: see you (11:45:38 PM) ***thunder waves