Author: Elaine (page 1 of 2)

The Importance of Community

(From a talk by Georgia candidate for Governor Stacey Abrams on the importance of speaking up)

A charming mix of TED talks, “lightning talk” presentations, business networking, and political rally, Lesbians Who Tech (LWT) is unlike other technology conferences I have had the privilege to attend.

The amount of joy makes it more like a party, even though it contains all the ingredients of a conference. The sense of community and camaraderie is abundant sometimes to a sickening (as in sugar) level, not to mention the marketing overdose, but it is necessary to provide (and fund) a safe space where the invisibles are acknowledged and heard. It’s almost like we are routinely ignored so often in such a matter-of-fact manner, that we can’t stop celebrating when given the chance to do so.

Even at the risk of tokenism and political correctness fatigue, it is generally favorable to encourage, support, and participate in the celebration of diversity, not just because it is decent but also because it is likely profitable.

Regardless of the ultimate motive of the decision to embrace diversity, at least the train is going somewhere: candidates are welcomed into the recruiting process, entrepreneurs get pitch opportunities, and businesses gains access to a non-typical talent pool while enhancing company image as an ally of minorities.

LWT is a kind of psychological compensation and a renewed call to action for a spirited subset of a sexual minority that is historically ostracized and only recently gained global traction as well as national legal rights in America, keeping in mind that these fragile rights are increasingly in flux under the current administration. Given this climate, it is unsafe to stay silent. Absolute objectivity is not always desirable, even for journalists.

Kate Kendall of National Center for Lesbian Rights quoted Elie Wiesel in her presentation on the importance of resistance:

We must always take sides. Neutrality helps the oppressor, never the victim. Silence encourages the tormentor, never the tormented.

Whinefulness, Perseveration, and the Infinite Loop

Why do people whine so much? Presumably, people whine about things they cannot control: a phenomenon, an action they must perform, or a situation they must endure, that are of an unpleasant and undesirable nature.

To me, “whining” is slightly different from “complaining,” in that whining contains an added shade of despair and futility. A complaint is actionable; a whine is useless and best ignored. Complaining is more effective, and whining is more impotent.

“Whinefulness” could be classified as an emotion, perhaps not as essential as joy, sadness, or anger, but definitely on par with resentment. In fact, whinefulness is the product of resentment.

I strive to stay away from whinefulness, both of others and of my own. It is functionless: it produces and changes nothing, transcends and transposes nothing, therefore it does not deserve my time. Repeated whining about the same issue or event without due action is an example of perseveration, a pathological behavior.

Perseveration is the repetition of a particular response (such as a word, phrase, or gesture) despite the absence or cessation of a stimulus. It is usually caused by a brain injury or other organic disorder.

This functionless behavior can be loosely described as an infinite loop, in computer science terms. Perseveration is different from Kierkegaard’s concept of repetition, which has a much broader temporal and psychological scope.

I have often found myself creating and entering infinite loops, but I have become increasingly aware of the early signs both in myself and in others. Effort must be made to prevent the creation of a loop, and boundaries must be drawn to avoid being brought into loops created by others. Upon entry, it takes more energy to exit the loop than the energy taken to create it. Since loop creation is already a waste of energy, entering the loop would result in a multiplied waste of energy.

It’s hard to believe human emotions can be reduced as such, but sometimes they can and should be.

Data Visualization Workflow

Extracted from an OpenVis presentation by Mike Bostock.

1. Prototypes should emphasize speed over polish.
Identify the intent of the prototype.
What hypothesis are you testing?
It needn’t look good, or even have labels.
Make just enough to evaluate the idea.
Then decide whether to go straight or turn.

2. Transition from exploring to refining near deadline.
Choose tools that facilitate transition to final product.

3. Clean as you go.
Be ruthless about deleting code.
You are a chef and the git repo is your kitchen.
You try twenty recipes before deciding on one.
Do you really want to clean up all the mess at the end?

4. Make your process reproducible.
Make a build-system that provides machine-readable documentation.
Accelerate the use of parts from previous projects.

5. Try bad ideas deliberately.
You can’t evaluate a visualization absent the data.
Don’t get too attached to your current favorite.
Don’t get stuck at local maximum; go down to go up.

JavaScript Closure With Dr. Hannibal Lecter

It took me a while to understand the concept of closure in JavaScript, but it can be easily visualized with a familiar analogy.

In police procedurals, suspects are sometimes brought into an interrogation room that contains a one-way mirror. The police officers¬†outside the room can observe the interrogation¬†that is happening inside the room through the glass panel, but the suspect and the interrogator¬†inside the room can’t see outside through the same panel.

In JavaScript, it’s the other way round. Someone inside the room can see what’s outside, but someone¬†outside can’t see inside. The objects¬†inside the¬†child scope (inside) have access to variables in the parent scope (outside), but objects in the parent scope have no access to variables in the child scope.

To illustrate this, let’s use The Silence of the Lambs, one of the best thriller films ever made. ¬†In this scene, ¬†FBI trainee Clarice Starling meets the incarcerated cannibalistic serial killer, Dr. Hannibal Lecter, for the first time.

In our absurd remake of the scene, Dr. Lecter’s cell has a one-way mirror where¬†he can see the outside, and those outside can’t see him. Clarice slowly walks down the hallway of the prison. She has seen Dr. Lecter’s files, but she has no idea what he sounds like, and she doesn’t know which cell he is in. As she walks by Dr. Lecter’s cell, he sees her, and says something to her.

The computer says inmate is not defined because Clarice, who is outside the cell (in the parent scope), cannot see the inmate who is inside the cell (in the child scope). On the other hand, Dr. Lecter, who is inside the cell (in the child scope), can see the agent who is outside the cell (in the parent scope).

Since Clarice can’t see who’s inside, she is not able to respond with a greeting.¬†She must enter the cell to see who she is talking to.

Now they’re talking. Clarice is terrified by the presence of Dr. Lecter, so she turns around and walks out, and Dr. Lecter follows suit.

We don’t know what happened to Clarice afterwards, but now we know a little bit about JavaScript closure.

Cloud Sentinel

Almost all new¬†software businesses these days are helpers. They offer software to¬†help you use other software or help you make other software. Compared to 20 years ago, it’s much harder to build¬†something¬†as fundamentally seismic to humanity as a Google or a Facebook. On the other hand, it’s much easier to make something that makes a selected group of people happy, thanks to open source culture. The wealth of resources and education available for free means that with a little talent, you can create¬†little earthquakes at a very low cost. Software monitoring is one of these rivers in the ocean of software business.

Software monitoring to software is just like the dashboard to your car. Your car dashboard tells you if you have enough gas or if something is wrong with your engine. Software monitoring tools tell you if something is wrong with your software and the things you use to run it. Monitoring had existed since software existed, but offering a set of integrated tools as a comprehensive service is a relatively new practice. In the past, developers would often patch together several tools and keep them in-house. More recently, some companies have begun to offer a full suite of monitoring tools, integrating them with cloud computing services, and topping up the package with advanced features in analytics. They offer this as a subscription, removing the need to host and maintain locally, and they have managed to make it look sexy.

The main¬†perk¬†is efficiency: it frees up local resources to focus on gaining insights from data. Incorporating nifty¬†visualizations and statistical techniques allows you to feel like you are doing a data scientist’s work rather than a system administrator’s. “Data scientist” is definitely¬†sexier than “Sysadmin,” which is why¬†companies market monitoring from a data science angle. What used to be a dreaded chore becomes appealing, and this is a side effect that is not to be overlooked. Having a reputation of installing ping pong tables in the office, making things fun is a staple allure of the software business.

The old model of monitoring is like having a horse carriage. You buy some horses (monitoring tools), a stable (servers) and a full staff to maintain the health of the horses. The new model is more like having a self-driving Iron Man suit with a supercharged assistant like Jarvis. He is more than a helper; he is a sentinel. He is an artificial intelligence that can auto-adjust your power mode according to flight conditions, verbally alert you to engine problems while cracking jokes, and make you green juice in the morning.

A fairly popular general model of software monitoring is Gartner’s¬†Application Performance Management, which is an apt description, but I’ve already started yawning. Words, names, and imagery do matter, a lot (sometimes even, or especially, punctuation). Data scientists used to be called¬†statisticians¬†or analysts. When someone thought of calling the job¬†a different name, granted that the field became associated with artificial intelligence, the same job¬†evolved into¬†the sexiest job of the century. There has to¬†be a more imaginative name¬†to software monitoring as data science is to statistics. Like Sentinel. Software Sentinel. Cloud Sentinel. Cloud Computing. Data Science. Cloud Sentinel. Got a nice ring to it, doesn’t it? A bit strange at first, but remember, cloud computing used to be a headscratcher not long ago, and now it’s become a beloved buzzword in Techspeak.

 

Explaining Cloud Computing With Game of Thrones

The other day, I was trying to explain to my parents what cloud computing was. They are intelligent folks that never had to incorporate computers into their careers whatsoever, so they are the kind of people who need help filling out online forms or signing up for social media accounts. You can’t expect them to understand any vocabulary of Techspeak, a professional dialect that technologists hold so dearly in their hearts for good reasons. I like to explain esoteric concepts¬†to people who don’t care about them, and see if I can make them care. This is a practice of the Feynman technique, which not only reinforces the concepts in my own mind, but also reinforces the reasons I should care about them in the first place.

Before telling the story, I want to talk about¬†semantics, which is the issue at heart in trying to explain technical concepts to the uninitiated. I am using “Techspeak” as a neutral definitive term, in a matter-of-fact manner, without any negative connotations. In every field, there is a set of vocabulary that defines the crucial concepts and functions to summarize¬†the existence of the field. When your doctor uses medical terms in Latin to describe your health conditions, you don’t mock them by saying they are using “Doctorspeak.” The use of those terms is simply an accepted convention, partly because they have been used for centuries, and possibly because Latin is more concise in its ability to explain so much in fewer words than plain English.

In the business world, the running joke is on “Corporatespeak,” describing empty¬†rhetoric that people use simply for dramatic effect, without any or much actual content in the communication. In other words, sometimes when people speak, what they are saying is not in the meaning of the words but the fact that they are saying something and the situation they are saying it in. Nevertheless, Corporatespeak in and of itself is just a dialect available for effective communication within a context.

In the world of computer science and technology, the vocabulary used is crucial to the operations at hand. I would argue that the significance of this language is similar to the way doctors use Latin: based on long-standing conventions and the utility of conciseness. Technologists trained in computer science share a common cryptic (to outsiders) professional vocabulary the same way doctors trained in medical schools share a common vocabulary, and that is the de facto language they use to communicate with each other.

Alright, back to explaining cloud computing to my parents. Let’s just start by thinking about having a business and running it. You need computers, obviously, even if you’re running a deli selling sandwiches and beer. You need computers to calculate and record transactions. For keeping track of inventory. To pay your staff. For a deli, maybe you just need one computer at the cash register for the transaction bit, and another computer for the rest. That’s not a big deal, because it’s easy to get and maintain two machines, and to teach your staff how to use and take care of them.

Now, let’s think of a more complex business: a filmmaking company that provides special effects by combining actual filmed footage with computer-rendered images, in order to create spectacular and imaginative scenes. Star Wars, Captain America, Cloud Atlas, Avatar type of stuff.

You’ll need storage space for all the image files. You’ll need servers to dish them out to you when you need them. You’ll need applications to process the images. All of this is written in computer code, transformed into electronic signals, stored in massive physical structures. This is where the hard part is: maintaining these monstrous physical structures and the code required to create the magic.

Not just the physical space you need to rent or buy to house them, or the electrical bills you run up for keeping them on, but also additional staff who know the ins and outs of the thing and how to fix it when something goes wrong. The bigger your company is, the bigger the monster is, and the more steps there are between different points. More things can go wrong.

It’s useful to think of this computer system as a physical monster: like one of those dragons in Game of Thrones whose owner, Daenerys, uses as fighting machines in battles.

As your business grows, you are likely to have to encounter these monsters. The essence of cloud computing is that you find¬†someone who owns these monsters. You pay her for¬†whatever tricks she makes the monsters perform for you: breathing fire to destroy a hundred ships, for example. You don’t have to rent¬†a big house to keep the dragons, and feed them huge amounts of food. Daenerys would do that.

So there you are. Cloud computing is like having Daenerys as an ally. She would release her dragons to help you win battles, for a modest fee.

Programmer Anarchy, a Post-Agile Model

Just came across a software development organizational theory called Programmer¬†Anarchy, or Developer¬†Anarchy, in a podcast by Software Engineering Daily.¬†Fred George and Antonio Terreno developed the theory back in 2010 as sort of an “Extreme Agile” guerilla-style model to trim fat from a software development organization.

The theory seems to have a Marxist flavor, in that it seeks to eliminate the roles of the capitalist and bourgeois in favor of maximum productivity. Just as a proletarian revolution creates a new kind of state that empowers workers, Programmer Anarchy creates a new kind of organization that empowers software developers, perhaps at the expense of workers of other functions in the organization. Parallel to the political analogy, this model probably works best for a company at its infant stage.

At its heart, Programmer Anarchy might be closer to¬†the idea of “creative destruction,” developed into an economic idea by Joseph Schumpeter in Capitalism, Socialism, and Democracy. The term “anarchy” has a very specific meaning in politics, although the specificity and intensity of the word has somewhat diluted with its frequent use in popular culture. The branding of the theory is sort of a hyperbole and a marketing gimmick; perhaps “Programmer Liberation” might be better in terms of sensitivity, but then again, the word “liberation” has its own controversies.

Individual angle: career development

Compared to traditional organizational structures, this model seems to foster a more holistic environment for individual career development. It promotes continuous learning by doing, and provides a high degree of freedom for doing so, provided that production goals are met.

This rings true for many modern professions, particularly in fields with rapidly changing landscape, such as media and technology. In the case of technology,  there is the constant race to the top for funding and market share. In the case of media, technology is responsible for shaking out a lot of media jobs, in the transition from print to digital.

“When your skill becomes commodity, and people put spreadsheets together and tell you what rates to charge,” George said. “This is when you lose your jobs to the offshore firms.”

Which is roughly how large corporations view their employees: a figure on a spreadsheet to be “streamlined.”¬†That said, only high-performing and highly committed individuals can benefit from the Programmer¬†Anarchy model. In this model, virtually all support roles (quality assurance, project management, business analyst) are permanently eliminated. Only those with high-level skills in software development will remain.

Organizational angle: efficiency

The traditional way is to keep people who are specialized at something to do only that thing, and absolutely nothing else. There is more friction against getting things done, and there are hand-off costs between specialists, because people have to spend time communicating to each other. The more people you have, this process repeats the more, and more man-hours are spent. Programmer Anarchy speeds up the workflow by removing the hand-off costs.

It dissolves the bureaucracy that sometimes prevents things from getting done. In a more traditional setting,¬†a non-developer internal client might rely on a developer to resolve a problem, and no developer would dare to do a thing until a “user story” has been entered into a Jira ticket. This means you need to talk to at least three “managers” before everyone would even know what’s going on.

Developments

Terreno, one of the authors of the original paper for the theory, reported mixed results in 2012, two years into the Programmer Anarchy experiment. George is still promoting the model pretty hard, and I’d be interested to see where it goes.

Autopsy: Hierarchical Clustering

What do normal people do when they want to rank a bunch of cities according to some features? Import the data on an Excel sheet, calculate a composite score across the features, and sort according to the composite score. What do crazy people do?

Why, a clustering, of course.

I had about 20 features of 100 cities, and I wanted to put them into groups according to similarities across the 20 features. I loaded the data into R and did a hierarchical clustering analysis. It works like this. The algorithm loops through all the observations, defines some sort of dissimilarity measure between each pair of observations, and spits out a tree diagram with a fancy name: a dendrogram!

Rplot01

Looks pretty intimidating. I felt very sophisticated. But I wanted the names of the cities at the end of the tree, and R just wouldn’t listen. What to do? After a few hours:

cluster

Hierarchical clustering is a type of “unsupervised learning,” which is what you do when you have no idea what kind of pattern or grouping you are looking for. It’s also a “bottom-up” agglomerative clustering. It starts by comparing individual observations, and works its way up until every observation is covered.

In terms of a tree diagram, it starts from the “leaves” level and works its way up the branches, combining branches up to the trunk of the “tree.” The algorithm usually measures dissimilarity in terms of “Euclidean distance,” which is a fancy way of measuring the distance between two sets of points in an abstract space.

OK. So I found that New York is really similar to Los Angeles and Chicago. All this work was basically useless, but it was kind of fun.

Louder, Faster, Harder, Stronger: The Loudness War

If you’re in the music industry or if you nerd out on music, you would have heard of the Loudness War.

The concept is fairly intuitive: the louder the song, the more likely people will hear it, especially when they aren’t paying active attention. Being loud helps a song get noticed when it’s on the radio during your cab ride or at the grocery store. If your song is louder than your rival’s, it gets the listener’s attention, and people are more likely to remember it the next time they are streaming or buying music.

The tradeoff is that when you make a song louder, you sacrifice sound quality. In the process of boosting the overall loudness of a song, all the sounds are made louder, including the softer sounds. The contrast between loud and soft sounds is lost, along with the nuanced interactions between different instruments and the vocals. Basically, it sounds more crappy and boring.

Why do record companies commit this atrocity? Because it sells. Music analytics company Next Big Sound analyzed the audio features of 32,310 musical tracks by 751 artists, and found that loudness is positively correlated with sales, other factors held constant.

loudnessSales

In this contour plot, the x-axis represents loudness, and the y-axis represents sales. The weird multi-layer shape in the plot is the “contour,” which tells you where the various tracks stand in their loudness-sales relationships. The color represents the density: the blue bit indicates that a lot of tracks are clustered in that area on the graph.

Most of the songs in the sample tend to be on the loud side, as the center of the blue bit sits near the right end of the x-axis. Most songs really don’t sell that much, so a large cluster of the tracks sits low on the y-axis. The tracks that turn out to be mega hits are at the upper right corner of the plot, and they are all on the loud side.

What does this tell us? Loudness is correlated with higher sales. The relationship isn’t linear though; it looks more like a roughly log-linear (exponential) relationship.

As a consumer, I’ve adapted. I’ll just listen to the shitty loud songs at the club. At home, I’ll put on some good old classical on a curated sound system.

U.S. Mass Shooting Fatalities Since 2014

shootings

And therefore never send to know for whom the bell tolls;
It tolls for thee.

– John Donne, No Man Is An Island

This chart depicts the number of fatalities in mass shootings in the U.S. from 2014 to 2016. You can see clearly that the toll from the latest shooting in Orlando, Florida far outnumbers even the sum of a number of past shootings.

Frank Bruni sums it up pretty well. This isn’t an attack on a minority subset of a population, but an attack on the “bedrock” of our society: the very idea of democracy, acceptance, and diversity.

How many of these incidents are still going to happen before something is done? Sadly, maybe quite a few. “And to actively do nothing is a decision as well.”

You can’t really say “I’m glad this didn’t/doesn’t happen where I live.” Just because it didn’t, doesn’t mean it couldn’t.

First they came for the Socialists, and I did not speak out‚ÄĒ
Because I was not a Socialist.

Then they came for the Trade Unionists, and I did not speak out‚ÄĒ
Because I was not a Trade Unionist.

Then they came for the Jews, and I did not speak out‚ÄĒ
Because I was not a Jew.

Then they came for me‚ÄĒand there was no one left to speak for me.
РMartin Niemöller

This is a work in progress, and the code is on GitHub. The data source is Gun Violence Archive.

Older posts

© 2017

Theme by Anders NorenUp ↑