So why would sendmail use IP over MX?

And so it goes that I spent most of the morning investigating  a pesky mail delivery problem which sort of goes as follows:

Running sendmail-8.13.8-8.1.el5_7 on CentOS5, with virtusertable and virtuser-domains. We (ourown.it) are handling mail for the domain customer.it:

  # dig ANY customer.it
   ...
  ;; ANSWER SECTION:
  customer.it.            9136    IN      NS      ns02.provider.com.
  customer.it.            9136    IN      NS      ns01.provider.com.
  customer.it.            31610   IN      MX      20 rigel.ourown.it.
  customer.it.            31610   IN      MX      10 alpha.ourown.it.
  customer.it.            3535    IN      A       10.213.221.92
  
  ;; AUTHORITY SECTION:
  customer.it.            9136    IN      NS      ns01.provider.com.
  customer.it.            9136    IN      NS      ns02.provider.com.
  
  ;; ADDITIONAL SECTION:
  ns01.provider.com.      3535    IN      A       10.227.46.9
  ns02.provider.com.      3535    IN      A       10.175.38.200
  alpha.ourown.it.                86400   IN      A       10.111.444.5
  rigel.ourown.it.                86400   IN      A       10.111.444.10

Customer.it is in virtuser-domains (so it *SHOULD* be recognized as local) but NOT in local-host-names (for long winded reasons I would rather avoid using local-host-names for this ). My expectation is that email should get to the highest MX (alpha) and be delivered there. What actually happens is that email sent to customer.it is queued up on alpha (even when it is sent form alpha itself) and delivery is attempted at the HOST customer.it (where no mail server is listed). This is what I see:

  #sendmail  -d17.99 -q -v -qRevit                                                 Running /var/spool/mqueue/r3C89pgp020280 (sequence 1 of 2)                     
  hostsignature(customer.it.)                                                    
  mxrand(rigel.ourown.it) = 148
  hostsignature(): getmxrr() returned 1, mxhosts[0]=customer.it.                   hostsignature(customer.it.) = evi... Deferred: Connection timed out with customer.it.

customer.it.                                                                    ... Connecting to customer.it. via esmtp...                                                                                   

MXs are correctly found:

  # echo "/mx customer.it" | sendmail -bt -d8.20
  
  ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
  Enter  

  > getmxrr(customer.it, droplocalhost=0)
  found localhost (alpha.ourown.it) in MX list, pref=10
  getmxrr(customer.it) returns 2 value(s):
          alpha.ourown.it.
          rigel.ourown.it.

I have many other domains thusly configured and working the crucial difference being that the IP for the domain is either not set or points to alpha.ourown.it. Address test outlines a difference:

  # sendmail -bt -d8.20
  ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
  Enter  

  > 0 aguy@customer.it
  parse              input: aguy @ customer . it
  Parse0             input: aguy @ customer . it
  Parse0           returns: aguy @ customer . it
  ParseLocal         input: aguy @ customer . it
  ParseLocal       returns: aguy @ customer . it
  Parse1             input: aguy @ customer . it
  Parse1           returns: $# local $: aguy @ customer . it
  parse            returns: $# local $: aguy @ customer . it
  
  
  > 0 alf@ourown.it
  parse              input: alf @ ourown . it
  Parse0             input: alf @ ourown . it
  Parse0           returns: alf @ ourown . it
  ParseLocal         input: alf @ ourown . it
  ParseLocal       returns: alf @ ourown . it
  Parse1             input: alf @ ourown . it
  Parse1           returns: $# local $: alf @ ourown . it
  parse            returns: $# local $: alf @ ourown . it

But its meaning is lost on me.

And then it struck me: the addresses that were being queued did not locally exist (it was someguy@customer.it rather than aguy@customer.it).

What happens is that sendmail, upon realizing that local delivery is not possible for the address at hand, "helpfully" tries the IP address as last resort. If that bugs you, as it bugs me, insert the line:

@customer.it               error:nouser 550 No such user here

in virtusertable, after all other customer.it records.

The strange world of SELinux and webmin

I am willing to share a few more sysdamin pains today, for the BDSM oriented people who enjoy suffering.

I am testing webmin on a CentOS 6.3 machine today. Purpose: creating a restricted web-paneled environmente where a basically skilled admin can add users, mailboxes, aliases and so on  without hosing the system or opening mile-wide holes into it. It does look doable so far except....

I want mail delivery done in Maildir-style folders under the user homedir. Procmail is the perfect tool for this, given the following /etc/procmailrc:

LOGFILE=/var/log/proc_log
DEFAULT=$HOME/mail/

(Hint: the trainling slash does the trick).

Enters webmin, breakage begins.  A webmin-added user, balf, cannot have mail properly delivered, while a adduser-added user alf, can. Ostensible reason being that, in the first instance, procmail is incapable of creating the mail/ directory. The logfile trace is self-explanatory as usual:

procmail: Unable to treat as directory "/home/balf/mail"
procmail: Locking "/home/balf/mail.lock"
procmail: Error while writing to "/home/balf/_D3D.N35ZRB.localhost.localdo"
procmail: Lock failure on "/home/balf/mail.lock"
procmail: Assigning "LASTFOLDER=/home/balf/mail/"
procmail: Opening "/home/balf/mail/"
procmail: Error while writing to "/home/balf/mail"
procmail: Assigning "LASTFOLDER=/var/spool/mail/balf"
procmail: Opening "/var/spool/mail/balf"
procmail: Acquiring kernel-lock

Uh? Google brings no joy other than the usual check permissions and stuff. Which I did. The gut instinct reaction (manually creating the mail directory) just pushes the problem deeper. Some work succeds in making the users exactly the same (same permissions, on home dirs, same group, same passwd and shadow entries, same files in the home dir. So it must be something with permissions, which means....SELinux. Which I usually turn off - but forgot on this time. 'ls -Z' verifies that the homedirs of the users have indeed different context:

 # ls -Z drwx------. alf mailusers unconfined_u:object_r:user_home_dir_t:s0 alf drwx------. balf mailusers unconfined_u:object_r:home_root_t:s0 balf

So, at this point you turn off SELinux, and all is well (put SELINUX=disabled in /etc/sysconfig/selinux).

On #donglegate, Adria Richards and disingenuous commentators

If you are even remotely involved in net.software, you have by now heard about Adria Richards, "donglegate" and the fallout of  what happened at PyCon. If not, you may read ArsTechnica's account , an almost legal opinion  here, and see a feisty, but funny, video here. I am also appending my TLDR summary of the story at the bottom of this post.

You should also know that:
1) I do not in the least condone the assorted rape and death threats that Ms. Richards has been the target of;
2) I also find unacceptable the privacy invasion she saw fit  to commit;
3) I think she is (partly) responsible for the developer's dismissal;
4) I think she was rightly dismissed for doxxing people;
5) I think that she is an embodiment of American puritanism at its worst, straight out of Hawthorne's novels.

The reason I am writing this (apart from getting it out of my system and turn to more productive thoughts) is an emerging trend about commentators.

Which is, writing a "fair and balanced" piece which starts by sort-of-acknowledging that

'yeah, in the beginning she may have been a little exuberant  but hey, look at the horrific - (and they truly are) - threats that she was in the end made a target of. That means that she was right, and if you dare be among their critics you are also a supporter of the rape-and-death-threat-extenders. And let's not get into all that pamby-namby stuf about privacy and public shaming, because it is bullshit  - "Red herring" or "ill defined" are the preferred terms -. Let us discuss instead the horrific blah sexist tech community blah blah please more page views blah thanks blah.'

Anybody sees a problem here? Because I do. Single sidedness aside, I mean.

The tech community has very heavily (with tweets, comments and likes) weighed in Ms. Richards'actions disfavor - with many women taking this position. No sane, reasonable post I read about this had anything favorable about the net.frenzy that ensued.

But the above commentators (no, I am not linking them, but they are easily found one is on the Guardian site) are conflating Ms. Richards actions and their aftermath with the death threats, using the latter to cast a favorable light on the former. Intellectually speaking, that sucks.

Which brings me to the third reason to write this which is urging anybody reading this (yes, all the three of you) and agrees with my views, to continue to actively defend them online and to avoid that this type of comments become the accepted wisdom on the episode and define what the response of the tech community should be.

Because if they do, it brings us a step closer to an Orwellian world where privacy and freedom of expression will be restricted to places not within remote earshot of an activist with a smartphone and an axe to grind. And that, also, sucks.

As for programming languages: you may want to stick to perl - its conferences are a saner environment :-)

Quick summary of the pycon incident

1) At the PyCon conference, Ms. Richards overhears (some would say "eavesdrops on" but I am willing to assume loudness on the part of the other involved parties) a *private* conversation during which juvenile jokes were being exchanged; (Jokes about big dongles, if you wanna know, as in - "Sure that guy has a bigger dongle than some other guy")

2) She tweets about it, *to a sizeable audience* and *with pictures* of the perceived abusers, taken *without permission* calling for a reprimend. Many would say this is as close to doxxing as it gets, and I would have to agree - I doubt anybody would disagree on the privacy invasion that Ms. Richards committed.

2a) The PyCon staff identifies the developers and tells them to stop  - which they do.

3) Ms. Richards writes a blog entry grandstanding on the heroic ("I know, you don't have to be a hero all the time") feat she performed for the greater good of the community ("The future of women in tech was on the line, and I acted" - no less)

3a) One of the pictured parties is fired (it'd be interesting - and not meaningless - to know the time ordering of 3) and 3a)

4) Ms. Richards is doxxed by some moron and (unforgivable) net.insanity ensues.

5) Ms. Richards is also fired, which -as an entrepreneur - I find is a reasonable course of action to take whenever blatant doxxing and privacy invasion are performed.

If the privacy issue seems of little import, here is some more food for thought: the smirking, bearded guys in the forefront of the by-now-famous pictures *are not* the dick jokers. I read elsewhere that common wisdom is that the person Ms. Richards took issue with is another developer in the background. So, while defending the future of woman in tech, Ms. Richards smeared a couple of innocent bystanders. I'm sure somebody will say that that is irrelevant, or even, justified collateral damage.

Turning off opportunistic client TLS in sendmail

It appears that sendmail, during mail delivery, tries TLS if the receiving server offers it. Fair enough, I can perhaps send over an encrypted channel, so I am going to try. Sounds harmless, right? Wrong.

See the follwoing SMTP dialog:

... Connecting to mail.snafu.com. via esmtp...
220 mhnicosi-unix.fubar.it ESMTP
>>> EHLO mx1.example.it
250-mhnicosi-unix.fubar.it
250-AUTH=LOGIN CRAM-MD5 PLAIN
250-AUTH LOGIN CRAM-MD5 PLAIN
250-STARTTLS
250-PIPELINING
250 8BITMIME

>>>; STARTTLS
220 2.0.0 Ready to start TLS
EHLO mx1.example.it
... Deferred: Connection reset by mail.snafu.com.
Closing connection to mail.snafu.com.

See the problem here? The receiving server shuts down the connection - possibly because it's not prepared to negotiate TLS with our server, sendmail queues the message and, on the next delivery, will go through the same motions in perpetuum.

I got the solution from this post of Mike Berggren. Basically, either stick

• Try_TLS: No 

(global) or

• Try_TLS:[host designation] NO

(per host) in /etc/mail/access.
Mike points out that adding the M=S flag to the CLIENT_OPTIONS macro in sendmail.mc has the same (global)) effect:

• CLIENT_OPTIONS(`Family=inet, M=S, Addr=1.2.3.4’)dnl

I did not try that though, as the first solution warks well enough, and my sendmail.mc does not use the client_options macro anyway.

I misteriosi misteri di Modena

Cara Gazzetta di Modena:

Volevo ringraziarti per la nuova, geniale rubrica umoristico-satirica
intitolata "i misteri di Modena". All'inizio avevo pensato si trattasse di una colonna seria, ed ero un po' perplesso (ad esempio perché mi pare che poche città - come dice anche mia moglie, che è di Roma - possano essere meno misteriose di Modena).

Solo nel corso della lettura mi sono reso conto della vera natura dell'articolo. E anche questo presentarsi come rubrica seria, in fondo, fa parte della geniale vis comica dell'autore.

E' solo osservando attentamente come il fondamentale disprezzo per le regole della composizione e della sintassi italiana (che a tratti coinvolge l'ortografia) diventi gradualmente assenza di logica che si possono apprezzare le vere e proprie perle umoristiche che l'autore ha disseminato in questi godibilissimi articoli.

Mi riferisco a frasi come:

(http://gazzettadimodena.gelocal.it/cronaca/2013/03/03/news/scarpa-professore-tiranno-o-scienziato-illuminato-1.6633821)
"Nato nel 1752 a Lorenzaga, paese vicino a Venezia, ha conosciuto Napoleone e divenne amico di Alessandro Volta. Culmina la sua carriera all'Università di Modena nel 1772."

dove il supremo, farsesco disprezzo della consecutio temporum si coniuga alla costruzione di "culminare" come verbo transitivo (io ti culmino, o carriera);

Oppure: "Ma qual era lo scopo del teatro anatomico voluto da Scarpa proprio a Modena? Lo scopo era quello di effettuare, davanti ai propri studenti, dissezioni sui corpi di cadaveri, per studiare gli organi umani e le funzioni vitali, tanto che diverse parti del corpo umano prendono il nome dato da Scarpa." dove una brillante tautologia (il teatro anatomico che serve a studiare gli organi umani:ohibò, chi l'avrebbe immaginato?) si uniscono buffe costruzioni (corpi di cadaveri dove chiunque scriverebbe cadaveri) ;

(http://gazzettadimodena.gelocal.it/cronaca/2013/03/10/news/villanova-nella-chiesa-dove-si-cacciano-i-demoni-1.6668450)
O anche "A fianco della normale attività religiosa al servizio della comunità di Villanova, il primo parroco, Don Goffredo Polacchini, fino al 1971 esorcista incaricato della Curia modenese, svolgeva anche le pratiche per sconfiggere le possessioni.....Ma sul perché Villanova fosse diventata il punto in cui il bene sconfiggeva il male, anche questo è avvolto dal mistero. Il freddo e la leggera foschia scendono sul campanile della Chiesa, mentre le storie di diavoli si perdono per le campagne, tra casolari e campi della pianura.", dove si sceglie di ignorare che, visto che il parroco di Villanova era anche esorcista, il mistero non c'è, e che le storie di diavoli sono davvero perse, visto che l'articolo non ne contiene alcuna. Per tacere della irresistibile parodia della retorica misterica che ispira la chiusa dell'articolo. Quasi ci si perde l'ulteriore sfregio alla grammatica (“Ma sul perché...anche questo è avvolto dal mistero”)

Il pezzo di Domenica 17/03, poi, sulla Chiesa di S. Domenico, è un vero gioiello:

"Un altro (mistero. ndr) riguarda i misteriosi tunnel sotterranei, e qui la storia si infittisce nei secoli, per tornare al 1243" (Bella la storia che si infittisce quasi fosse semolino.);

"Lo scopo era quello di mettersi al riparo da attacchi nemici al Ducato, ma anche per andare o uscire dalla città per diversi motivi come [...] commesse segrete nelle ore notturne" (Raffinato il passaggio da "scopo di" a "scopo per"; arguto il finto contrario "andare o uscire", enigmatiche le commesse segrete - forse sono le impiegate dei negozi del centro?)

Ma davvero non è possibile elencare il delirio di strafalcioni, anacoluti, antinomie e assenza di generale buon senso che, presentandosi quasi ad ogni frase, rendono questi articoli degni degli indimenticati fratelli de Rege.
Mi permetto solo un suggerimento: spostate la collocazione della rubrica in fondo al giornale in modo che anche i meno avveduti possano coglierne la funzione umoristica. Magari al posto di quella triste rubrica chiamata "Figadein", che - abbastanza inspiegabilmente - immalinconisce periodicamente la lettura del giornale. A proposito: qual è la sua funzione? E' un po' che me lo chiedo: questo sì, che è uno dei misteri di Modena.

On disappointment - Nassim Nicholas Taleb

I have - in the past - been greatly enthralled by an author called Nassim Nicholas Taleb and by his writings (he wrote a book called "The Black Swan", among others) and ended up having a very high opinion of him. Then, rather uncharacteristically for myself, I joined his FB page.

That was just the cure I needed to remind me about separating fact from fiction and expectations from reality when it comes to humans. While it was obvious - from his writings - that the man has vaste personality problems, I had somehow conceived the idea that shared intellectual values would overcompensate for that. Wrong.

Taleb - the internet person at least - appears to enjoy the totally acritical adulation of a crowd of fanboys that practically swoon on his every utterance (if I were him, I could hardly resist to write now and then things like "Pass the salt"  or "Basketball shoes are really comfortable" and watch the waves. That possibly explains my shortage of fanboys).

Apparently endowed of a very short fuse, he proves to be very intolerant of differing opinions and absolutely incapable of acknowledging any misstep in his thinking. And, he makes several. In one recent post he absurdly likened the Roman Empire to the Sicilian Mafia - showing extreme poor understanding of both. Confronted by the annoyed reaction of some followers, he proceeded to:

1) refuse to admit he had written a piece of nonsense
2) restate it several times without further explanation, in the apparent attempt to test the temper of his opponents, while also calling them "rude". ("This is not an argument, it's contradiction" "No it isn't" "Yes it is" ""No it isn't" - Monty Python)
3) avail himself of a lifeline thrown to him by one of his supporters in a "My point exactly" sort of way. Too bad its point had been entirely different, not to mention hard to fathom. In any event it appeared to be a - rather childish, coming fron an ethnical Greek -  attempt to establish the moral superiority of ancient Greeks on ancient Romans.
4) proceeded to unfriend the opponents and expunge their rude (i.e. disagreeing) comments from the thread. ("I had chip on my shoulder, kid, and you just knocked it down")


The last straw - as far as I am concerned - is this amazingly hypocritical piece of prose, which I append for the comfort of any purely hypothetical reader (hi, Nassim!) and as a prevention against acts of the Ministry of Truth (cue Orwell):

"Friends, I need some help correcting a distortion.
When you "call a fraud a fraud" (that is the members of that 1% that-cause-harm-without-skin-in-the-game) the strategy has been to turn your message into its exact opposite, something misanthropic ("if he hates me, an economist-journalist-fragilista-modernity advocate, it means he hates everybody"). Or "if he hates modernity, he is a haughty elitist" (the exact opposite of the true message that holds the nobility & independence of "those who make a living stanfing up or lying down".)
The corruption of the message has been largely controlled with "one of my books' title". But people are still doing it with "another of my books' title".
So I would welcome some contribution to the comments to dispel the cognitive dissonance there. Thanks!"

In which Nassim asks his followers to  provide favorable reviews on Amazon for his books, with tasteful code words ("distortion","cognitive dissonance"). His lovers followed quickly suit, stuffing the Amazon page as required - with nary a lone dissenting voice, this time. My reaction was to go to the page and write my actual opinion on the book in question - which is, that I didn't like it very much , and considered it vastly inferior to his other books (which I have favorably reviewed in several occasions). That did of course get me unfriended in a nanosecond - what a surprise. (No, it did not hurt at all, but thank you anyway)

And this makes it for me. Although someone might considered this pettiness a minor sin, I see it  as a capital flaw coming from someone that regularly displays an amazing high handedness on subjects such as ethics, wisdom, intellectual integrity. In fact I find it destroys just about every single non technical bit in his books, which I will find quite hard to reccommend henceforth. It also validates a great deal of what his harshest critics tell of him, but that's a different story.

Well, 'sic transit gloria mundi', I guess.

Antique Iron

An IBM 029 keypunch

This is a picture of a keypunch akin to the one I typed my first lines of codes on:













      PROGRAM SOMMA
C
C Calcola la somma di due
C numeri e stampa il 
C risultato 
C

      INTEGER A,B,SOMMA
      A=1
      B=3
      SOMMA=A+B
            PRINT(6,100),A,B,SOMMA
            STOP
 100      FORMAT(I4,1H+,I4,1H=,I4)

That' FORTRAN 66, Bay bee.

The twisted. The perverse. The developer.

Say you are running an android emulator for development reason and you want to copy some text to the (emulated) screen. That would be a CTRL+V on most platforms/applications.

But we developers have better ways, just:

# telnet localhost 5554
sms send 123123123 The text you want to copy
quit

You send an SMS to the phone, then in the emulator, open the message, copy the text, and paste wherever you want. Get it? Ain't that just beautiful?

This is the brain of developers laid bare for your wonderment. I kid  you not.

P.s: hats off to Stack Overflow for the tip
 

Custom column names on Django ManyToMany fields.

Adapting Django  to legacy databases requires you to control table and column names.

manage.py inspectdb does a nice enough job of it (but beware of missing to_field on ForeignKey relationshiops when not linking to the primary key). 

ManyToMany fields, however, are left in the cold. Table name can be set with db_table=, but there is no (documented) hook to influence column names. Finer control can be achieved through the relatively new 'through' argument: that, however, kinda ruins many of the ORM language and of the admin application capabilities.

I googled myself blind and visited a few blind alleys. Postgres, for instance, does not have updatable views, or one could use a view to sort of rename columns - without actually doing it - thus appeasing django's engine; updatable views could be emulated with a CREATE RULE, but I decided against going there.

I dived in django code - the related.py and construction.py files - not pretty .I will heretofore direct to those files anyone extolling to me the clarity of python coding.

I eventually came across a Django Snippet that the author appears to have  retracted, but that the internet wayback machine still preserves. It did still require a modification, but I finally came up with:

from django.db import models



class CustomManyToManyField(models.ManyToManyField):

    def __init__(self, *args, **kwargs):

        source = kwargs.pop('source_db_column', None)

        reverse = kwargs.pop('reverse_db_column', None)

        if source is not None:

            self._m2m_column_cache = source

        if reverse is not None:

            self._m2m_reverse_column_cache = reverse

        super(CustomManyToManyField, self).__init__(*args, **kwargs)



# later....



class TipoCommesse(models.Model):

    id          = models.IntegerField(primary_key=True)

    descrizione = models.TextField() # This field type is a guess.

    tipo_fasi   = CustomManyToManyField('TipoFasi',db_table='t_fasi4t_commessa',source_db_column='id_tipo_com',reverse_db_column='id_tipo_fase')

    def __unicode__(self): return self.descrizione

    class Meta:

        managed =  False

        db_table = u'tipo_commesse'

        verbose_name_plural='TipiCommesse'

Note the managed = False line in the Meta inner class indicating that syncdb & Co. need not alter the database for this type. Besides being a sane precaution to set on everything when dealing with legacy DBs, it is doubly necessary here because finding out how to persuade Django to honor the column specifications when actually creating the tables proved to be beyond my ability - that's the above mentioned - and by now infamous - construction.py file. The effect is that (as can be seen with manage.py sql, with manage=True) Django would still generate columns of the form re11rel2_id even for the custom m2m field.




Also note that this trick is fragile: it depends on django actually using the 'column' and 'reverse_column' names when currying the attribute accessor (don't ask). This has already been broken once in the past.


Finally, a big 'Thank You' to Quentin Tarantino for making Django searches a more interesting experience with his "Django Unchained" release ;-)

Isole Venete

Il burino di Burano,
Su un muretto qui a Murano
mentre pensa alla Morena
e, col viso un po' murino,
mangia mesto una murena.

Se ne andrà forse a Mirano,
il burino di Burano,
ben lontano da Murano,
per dimenticar Morena.

--Ettore Sacrestia.