kaberett | A snippet from last night

From:

alexseanchai

Hee

From:

ex_we935

That is sort of brilliant. :3

~K.

From:

birke

Indeed. :D

From:

ofearthandstars

Hee!

From:

untonuggan

Hee.

From:

swaldman

Heeeee!

(also, yes, I do this. I write a script to plot something, then somebody points out it would have been straightforward in Excel...)

From:

simont

The difference, though, is that if you ever need to do the same kind of plot again on new data it's a one-touch operation – just haul your script back out of source control and run it again – whereas the 20-30 clicks and menu item selections you would have used to do it in Excel will all have to be redone from scratch.

Also, in the case where it's not really simple in Excel, the Excel user will tinker around for ages and eventually manage to get it right, and may not even remember quite how, and would have almost as much trouble doing it again, and yet they will still somehow feel that they got the better of the bargain over the person who wrote a Python script, because it would have taken them more effort to learn Python first.

Programmers are people who reliably recognise both those phenomena as false economies of time...

... whereas Excel users are people who reliably recognise as a false economy the learning of Python and careful writing of a nice reusable script when you weren't ever going to need it again :-)

From:

emperor

I was thinking something like this, but you put it much better :)

From:

swaldman

Oh, absolutely! But... yes, your last point. For a rigorous, repeatable analysis, it's absolutely worth scripting it (and maybe even using something like Sweave, or Kintr, or Matlab's clone of them, to document it). For quickly plotting something to see what you've got, while still messing around with refining the method, copy and pasting to Excel will often be the better option.

I think it's reasonably analogous to people who spend two days optimising a program so that it takes 10 minutes rather than 30 minutes to run, when they're only ever going to use it a handful of times.

From:

simont

spend two days optimising a program so that it takes 10 minutes rather than 30 minutes to run, when they're only ever going to use it a handful of times

Mmmm. Though one should not overlook the fact that you also run the program a zillion times during development as well as the handful of runs on the real data after it's finished, and those test runs count too!

I once had to write a program to analyse a large data set, and on the basis of 'do not optimise prematurely', I wrote it in Python rather than C, and the half-written versions tended to take about an hour to run on even my test data set. After two days of fixing one bug after another, I decided this wasn't working, and rewrote it in C with properly fast data structure design – and saved development time, because the effect was that now after fixing each bug I only had to wait 30 seconds instead of an hour to find out where the next bug was.

Incidentally,

kaberett, sorry to have derailed your double entendres with a software engineering subthread :-)

From:

kaberett

Given that I have successfully managed to derail sex I was having via discussion of APIs, I'm not complaining ;) (It's interesting!)

From:

simont

derail sex I was having via discussion of APIs

I feel there are probably a zillion predictable jokes at this point involving things like re-entrance and use of objects, but perhaps I'll try not to splurge them all out in one go :-)

From:

kaberett

:-) (Hello you!)

'course I will use it again, and more than that I'll use the skills I pick up in writing it again -- and as below given that the alternative is faffing about tidying the thing up manually in the GIMP...

From:

simont

Urgh, yes, once GIMP gets involved you're almost certainly in the space where it's actually less effort to write the program even if you only needed to run it once!

On this general subject, actually, shortly after I published The Descent to C, I had an email from someone asking about how I'd drawn the diagrams. I replied with several paragraphs of waffling about PostScript, ImageMagick and Perl scripts, and was quite apologetic about that since I expected he'd probably turn out to have been looking for a recommendation of a GUI drawing package. Fortunately, he was a programmer type himself, and it turned out he'd been rather hoping that I hadn't drawn them all by hand in Inkscape or similar, so that he'd get useful scripting tips rather than a GUI recommendation. Phew :-)

From:

kaberett

heeeee nicely done :-)

From:

hairyears

Yes, this.

But what's a 'programmer'?

Me, I'm someone who pointed out that over half* the desk time of the entire office is spent moving data into spreadsheets. Mostly by copy-and-paste, sometimes by Excel's built-in functions; and the error rate is horrifying.

So I wrote a drag-and-droppable sheet that does this in three find-the-file clicks, or a simple one-click 'repeat'. It does it reliably, it checks that the full extent of the data is covered by the existing calculations, and it journals who, what, and when.

Any file, including any Excel file; any database; any tabulated data on a web page...

It's also nice to look at: clarity and good formatting, logically-laid-out controls with labels and popup 'deep description', and the departmental logo.**

The users who got it love it.

Departmental IT took the idea -and I gave them the 'prototype' - and came back with a clunky and ugly and inflexible special spreadsheet.

Nobody's using it until they are compelled to. And 'Programmer' means that outcome, and the people who do it, to everyone in the department who hasn't got the prototype***.
.
.
.
.

* That's not an estimate. I do observation and measurement.

** Yes, I'm a UI designer. Who knew? HCI's a big thing at DeMontfort. But the tech teams here do not regard girly decorating stuff as something a 'programmer' would or should be interested in.

*** Yes, I do testing. Unit testing, regression, and observed UAT. And I document it. That makes me a software engineer, rather than a programmer****.

*****One of these two terms means 'I write reliable programmmes.

From:

hairyears

Yes, this.

But what's a 'programmer'?

Me, I'm someone who pointed out that over half* the desk time of the entire office is spent moving data into spreadsheets. Mostly by copy-and-paste, sometimes by Excel's built-in functions; and the error rate is horrifying.

So I wrote a drag-and-droppable sheet that does this in three find-the-file clicks, or a simple one-click 'repeat'. It does it reliably, it checks that the full extent of the data is covered by the existing calculations, and it journals who, what, and when.

Any file, including any Excel file; any database; any tabulated data on a web page...

It's also nice to look at: clarity and good formatting, logically-laid-out controls with labels and popup 'deep description', and the departmental logo.**

The users who got it love it.

Departmental IT took the idea -and I gave them the 'prototype' - and came back with a clunky and ugly and inflexible special spreadsheet.

Nobody's using it until they are compelled to. And 'Programmer' means that outcome, and the people who do it, to everyone in the department who hasn't got the prototype***.
.
.
.
.

* That's not an estimate. I do observation and measurement.

** Yes, I'm a UI designer. Who knew? HCI's a big thing at DeMontfort. But the tech teams here do not regard girly decorating stuff as something a 'programmer' would or should be interested in.

*** Yes, I do testing. Unit testing, regression, and observed UAT. And I document it. That makes me a software engineer, rather than a programmer****.

*****One of these two terms means 'I write reliable programmmes.

From:

kaberett

So for me the deal is that in order to get plots my supervisors're willing to let me consider publishing via Excel/Calc, I have to first plot 'em up then export the chart as an image then tidy them up a lot in a graphics editor, whereas matplotlib spits out stuff they're happy with first time. And I'd much rather be learning to code than fucking about manually tidying up every plot for the rest of my natural...

From:

swaldman

*nod* I'd struggle quite hard to get anything I'd think of as publishable from Excel.

From:

chiasmata

<3

From:

kaberett

:-) Thank you!

From:

vass

I would laugh at you, but I spent a nonzero period of time in the last week writing a Python script to poke fun of my favourite character in my current fandom. And testing it until it ran. At least your supererogation is PhD-related.

From:

kaberett

AHAHAHA I SHOULD FINISH THE ELEMENTAFIC I AM WRITING SO YOU CAN FACEPALM AT ME

(it is so far 750 words of fic and I have so far got to NEITHER of the points, which are (1) Sherlock was wrong about mass extinction events and (2) ... this one 1984 paper I came across, okay)

(I have a horrible feeling it's going to end up >5k)

(... it is at least probably better than the 600-word fic that's an extended fisting joke >_>)