War on Spam

Greg Linden, an ACM blogger, recently asked if the war on spam has been won. He wrote,

Today, e-mail spam appears to be a solved problem. A 2003 study External Link put response rates at 0.005%. A 2008 study External Link where the authors infiltrated a major spam botnet found response rates had fallen to under 0.00001%, only 28 sales out of 350 million messages sent. Spam filters appear to have forced down response rates three orders of magnitude in five years. Spammers have fought back with misspellings, adding additional text to mails, trying to customize each e-mail sent, and many other tricks to evade detection, but their increasingly complicated efforts have not been able to outwit the filters.

Similarly, Daniel Hamermesh, a Freakonomic Blogger, wondered where are his Viagra spam.

I haven’t gotten one of these in a year, after often getting several a day. I assume that the spammers realized that the return per period of time the price of the activity was less than its marginal cost: the opportunity cost of their time. They have shut down the business and moved to other activities that might yield higher returns.

Today, over 90% of the messages a typical company receives are spam. Bigger companies or government agencies get more than 98%. Spam is the AIDS of email. There are some effective treatments, but it is far from being eradicated.

Seen the movie You Got Mail? Everybody likes to to receive messages. They publicize their email addresses to get more messages. They register for services, subscribe to mailing lists, or comment on public sites. They also participate various forwarding schemes, like chain letters, memes, and many urban legends. Was it was great to get the free dinner coupon at the nice restaurant?

Statistics makes the problem difficult. With today’s spam volume, a 99.8% capture rate gives a typical user less than one spam message a day. When the capture rate is 99.6%, a mere 0.2% decrease, half of the inbox will be spam.

Only economy will win the war on spam. As long as sending email is free, there will always be spam. Before that changes, demand your ISP to have a quality spam blocker. Make sure your company stop spam at the gate. Strengthen your defense with a good email client that has a spam filter. Lastly, before yelling at your spam solution provider, check if you actually asked for those messages yourself. Try as they might, those software engineers cannot know that you changed your mind and don’t want the restaurant promotions anymore, at least not until you are in the mood for a nice juicy steak again.

Posted under Peek into my mind by sinyaw on Tuesday 31 August 2010 at 9:36 pm

如果你已經20歲了,你真的輸不起了.

傳聞李開復的文章,真是洛陽紙貴.網上流傳不止.Google一下,不難找到轉載. 年輕的被說的心悸,過來人看得心有戚戚焉.

同樣的文章,古今中外,歷史上有了幾千版了.李開復的威力,在他的踏實,明確.(但李本人說這文章不是他寫的.) 幾句:

如果你20歲以後所花的每一分錢還都是伸手向父母親人要來的,那你的滿身名牌就只能襯托出你的無恥.

不要與浪子,文藝青年交往,別和沒心沒肺的人在一起,別和沒有正當職業混日子的人在一起.

說穿了,要務實獨立,不要夢幻痛苦,也別想一步登天.

但真想想,沒有幾個天才20歲就念完書了.基本的大學文憑要22歲.這一代近30才走出校園的,比比皆是.我大慨可以接受把他寫的加個10歲.三十以前,人追求的是應該是理想及愛情.在那追求的過程中,年輕人走過那世世代代都走過的路,才站立成人.沒有那段夢幻痛苦,怎麼破繭成蝶?

人的成長歷程,一定有一段是在找自己.找到後,才可能走下一步.過來人看到下一代的歷程會心疼,”能不能聽話,別去找了,你就在這兒.” 很不幸,每一代都必須過這關.

每一代,都有一群過不了這關. 那就是被時代淘汰掉的一群. 如果自己的孩子在那群裏,當然心疼. 看到明明有天分的被淘汰,也心疼. 那些孩子,只要聽話,就能好好的,多可惜.

其實,不聽話的年輕人,當然淘汰率高.但把人類推到新高的偉人,也是年輕時不聽話的那群.

順其自然吧!

Posted under Books & Reviews, China, Peek into my mind by sinyaw on Tuesday 24 August 2010 at 11:00 pm

The End of an Industry

B&N

Barnes and Noble gave us lots of joy when the kids were growing up. Our normal source of books was the library. Each trip meant scarily large book bags filled to the brink each way. Unlike the library, Barnes and Noble had new books! Kids would buy books that they have already read. The new books that they can keep forever were so exciting.

The Harry Potter series brought memorable years associated with Barnes and Noble. We would wait for a few days for the crowd to thin down. Then there was the pre-determined reading order: younger daughter would devour the book first, older one followed, and daddy, the slowest reader, last. Kids would wait, in patient agony, for me to finish before talking about it. We all obeyed the code: never ruin someone’s good read.

Exploring the aisles was a joy. I would stroll over to the geeky sections: SciFi, software, managerial, woodworking, gaming, etc. They would disappear into the fiction area or the stationary corner. We would all linger at the bargain bins, calendars, greeting cards, book lights, bookmarks, etc. In Beijing and Taipei, where online book ordering is less popular, I went to the giant bookstores (王府井書店, 外文書店, 誠品) just to relive the exploration. (Look. There is the Chinese chess session. Give me a minute.)

Amazon changed everything. Gone are the days we scrutinized the jacket, read several pages, and took a gambit to invest part of our lives into a book. We no longer discover books. We manage our reading lists. Reading used to be about getting lost into another world. Amazon made that world a bit smaller. That’s really why the business of Barnes and Noble, and all other bookstores, declined. Collectively, people spent less to explore the world of books. The value of aisles diminished.

If Border’s experience repeats, Barnes and Noble won’t find a buyer. Like music and movies, the ink-based industry needs to transform into something else. No doubt I will experience this transformation. In the mean time, I will click on my browser and refill my reading queue.

Posted under Peek into my mind by sinyaw on Tuesday 17 August 2010 at 4:16 pm

Hardiness

While the world watched the video feed showing oil guzzling out from the bottom of Mexico Gulf, a small pipe bursted near Dailian, a seaport on the north-eastern corner of China. The amount of oil was minuscule compared to the BP disaster, but nature does not really need much to be ruined.

The Big Picture blog, one of my favorites, showed stunning pictures. This is one of them.

This reminds me all those millions of migrant workers that built modern China. This is a country that millions usually just shut-up and do the jobs. They come home dirty, tired, still poor, and with the jobs done. Farmers, in every countries, are like that. They go out in the morning, do a day’s hard works, and come home at the end of the day. Jobs were done, no need to talk about it. Let’s just eat and have a drink to that.

What really made modern China!

Posted under China, Peek into my mind by sinyaw on Friday 6 August 2010 at 10:23 pm

More teachers, not less

Easter Island

In Collapse, Jared Diamond explored the possible reasons civilizations chose the path leading to their own demise. There were societies that lived in plentiful and nature will provide them forever. Yet, without external interferences, such as natural disasters or foreign invasions, they vanished. Frequently, they chose a ruinous short-term project at the expense of long-term viability. Spare no expenses! Let’s build a glorious monument. Alas, those monuments depleted the very resources that made those society rich enough to have such monuments.

Americans are making very similar decisions. A decade or so ago, they borrowed from the future so that they could have a easier time. When the future came for redemption, in the form of sub-prime crisis. Americans chose to borrow more from the further future, in the form of stimulus bail-outs. Future, after all, is not our problem. The next president will deal with it.

The most horrible choice is cutting education. Every states, every school districts, is firing teachers and reducing education budgets. Next decade will see under-educated Americans competing in the world defined by knowledge. They have no chance to win.

To worsen future competitiveness, Americans took on a harsher stance against immigration. Immigrants are the young, the strong, the smart, the driven, the competitive, the survivors. Those are the ones who sailed across treacherous sea for a better life for the family. Those are the ones who braved the wilderness to pursue gold in the west. Those are the ones Americans are turning away. “We don’t want you,” said Americans. “Because we don’t want to share with you.” Americans forgot that most of the wealth they have today were created by first or second generation immigrants to begin with.

Let’s see. This is the society that stole from its future. It sabotages its children with less education. It closes the doors for new comers that will make them stronger. And it now worries that their children will live shorter and poorer lives than themselves.

Oh yes, it also thinks that whoever think differently must be wrong or stupid.

Posted under Peek into my mind by sinyaw on Tuesday 1 June 2010 at 1:19 am

Who’s pays for speed?

In a simplistic way, the Net can be divided into three parts: the contents or services owners, these are the brands you know Google, Facebook, WoW, CNN, etc. They are the reasons you are on the net to begin with; next is the mystic cloud that magically connect those contents to you; last is the device you use to access those contents — that’s your smart phone, laptop, or set-top boxes.

Pundits proclaimed that the Net will have much higher bandwidth in the future. Google and Cisco both added fuel to that roaring fire of enthusiasm. It shall outpace Moore’s law, they said. Whatever speed you are connecting at today, you shall have 10 times, or even 100, more in 3 years.

Really? Who pays for the higher bandwidth?

Google does. As they are rolling out bright cables across the country, all they asked is a nominal fee not higher than your current bill. The new and fatter pipes are part of Google’s promotional budget.

Or Cisco will? They just announce the product that will change the world. (Juniper, of course, thought little of it.) It is a next generation router that handles 3 times the traffic than the previous one. As backbone operators upgrade their equipment, the pipe just got fatter. Isn’t it wonderful that technologies give us better life without us having to pay for it?

The paper-based industry pays for it. As online media replaces traditional ones, the revenue shifts. When New York Times becomes newyorktimes.com, its massive print press, truck fleet, and ad sales force disappear. The new company (probably under the old management) retains its old readership, or even larger, but with a much leaner and smaller operation. That pays for the new fat pipe.

You pay for it. Have you looked at your bills? How much you paid for your fixed-line phone, cell phone, data plan, cable TV, broad-band connection, iPhone apps, Netflix subscription, iTunes music, SecondLife spending, WoW weapons, etc.? Did you also upgrade to a faster connection? Did you even blink for the $15 Internet fee at the hotel you last checked-in? Internet is far from free to us surfers. The debate on how to monetize Internet has long ended. Consumers pay billions of dollars (and so did corporations) for their rich Net-based experiences. Oh, did you buy that digital camera by clicking through some links?

Yes, Internet will continue to expand and the bandwidth will continue to increase, like Google and Cisco have led you to believe. As for me, I am just trying to make a buck.

Posted under Management Thoughts, Peek into my mind by sinyaw on Sunday 21 March 2010 at 7:38 pm

Repo 105

Lehman Brothers’ executives had the incentive and were creative. To receive big bonuses, they needed to reduce debts. It was not easy, since the fundamental was bad. The right thing to do — like eat healthy and exercise — was hard and, more importantly, would not keep them up with the Joneses. So they cooked up a scheme that turned debts into transactions: same business, different label. The firm appeared healthier. This practice, now infamously called Repo 105, continued until the hidden debts reached over 50 billion dollars. Stopping would feel like an addict deprived of drugs; no one has enough will power to do so voluntarily.

Those executives were smarter than law makers and judges. It is unlikely that any of them will go to jail. Given their business savvy, I doubt any government can touch their vacation homes on Mediterranean coast stocked with extensive wine collection. When smart people lost their morality, there is only one thing left to do.

Corporal punishment! Public caning comes to mind. Of course, the same smart people would have lawyers arguing those being cruel practices. Did I mention that they were rich too?

Guess there is nothing left to do.

Posted under Peek into my mind by sinyaw on Tuesday 16 March 2010 at 10:05 pm

Where have all the young girls gone?

In the haunting song, Where Have All The Flowers Gone by Peter, Paul, and Mary. One of the verses wonder where have all the young girls gone. Of course, the song is about anti-war and the girls were mourning their dead boy friends. It appears that many Chinese young men, and their Indian counter-parts will soon wonder where to find eligible brides in the world.
Gendercide

The Economist reported that China will have a surplus of eligible bachelors as many as the entire Germany’s population! India, South Korea, Taiwan, and several mid-eastern countries are heading the same imbalance. This article reminded me the short-fiction Goddess, by Linda Nagata, in which future Indian elders routinely implant a gender selector in young woman’s wombs — have a boy or have no child what-so-ever. Modern Chinese and Indians would depend on the skills of the ultrasonic technicians for such decisions.

A surprise consequence is the size of the dowry. As eligible maids become less available, they command higher dowry or whatever forms of payment from the groom’s side. Parents of young boys need to save more, since wealth is part of the bidding to win a bride.

The society will fix this problem by itself. The surplus men will be denied opportunity to have a traditional family. They will share a wife with someone else, import a bride from outside, or stay single. The society will gain its gender balance in one or two generations no matter what. The question is really how violent the process will be — unwed young male is the source of most mayhem in the world.

Maybe the song is hinting a solution after all? Where have all the young men gone? Gone to graveyard everyone. Oh when will they ever learn?

Posted under Books & Reviews, China, Peek into my mind by sinyaw on Monday 8 March 2010 at 12:57 pm

Ethics

Kevin Smith

Recently, Kevin Smith, a well known Hollywood director, was kicked off a flight for being too fat. He was furious and started a twitter storm. His flamebuoyant communiques aroused media coverage and strong opinions everywhere. Fat people have rights too! They proclaimed. Besides, the number is on their side. The US shall have more and more fat people. Airline had better cater to their needs, lest losing their patronage.

Airline coach seats are about 18 inches wide with 32 inches “pitch” (the distance between two rows of seats). They are good for skinny and short people. (An average American woman has a hip width of 19.7 inches, and man 17.2.) Strapped in one of them is a discomfort that most people wish to get over as quickly as possible. It’s torturous if your seat-mate is nosy, noisy, smelly, messy, or, the worst, fat. Few things can bring up more terror than being squeezed against a stranger’s skin in a tight space.


A group of Californian students are seeking reversal of proposition 209, passed in 1996 to prohibit public institutions from considering race, sex, or ethnicity. Under this law, the UC system would consider only academic merits — GPA, SAT scores, etc. — to admit students.

In 2007, Latino, black, and Native American students comprised 45.1 percent of California’s high school graduates but those groups comprised only 16.9 percent and 19.9 percent of new freshman admits at UC Berkeley and UCLA, respectively. This is an outrage and a social explosion waiting to happen.

Prior to Prop 209, UC has the infamous affirmative action system that give each ethnic group a quota in the new admission pool. The graduation rate of those disadvantaged ethnic groups were miserable. UC system wasted precious resources, and admission slots, for those academically ill-prepared students that essentially robbed the education opportunities from those who were otherwise qualified.


In the study of ethics, there are the concepts of just, utilitarian, fairness, and equality. It is not just to violate any individual’s rights. But utilitarian will strive for the most good for the most people, usually sacrificing the minority. Kevin Smith may have his right as a passenger, but many more passengers suffer if airlines accommodate too much for fat people, directly with discomfort or indirectly through delay, lost of available seats, or increased price from reduced capacity.

The group that sued UC called themselves “Equality by Any Means Necessary.” Equality is an economic concept of the allocation of resources. (Five dollars for you and five for me will be equal.) Fairness means all considerations be merit-based. (We each get paid by how much we contributed will be fair.) It is obvious that UC admission system has not been equal. But is it fair? To answer, ask if the admission is merit-based.


Ethics is a complicated subject. Topics are controversial because different approaches do not arrive at the same result. We would like things to be just and good for the most, fair and equal too. Choose your stance based on an ethic yardstick, not whether you are fat or slim, Asian or Latino.

Posted under Management Thoughts, Peek into my mind by sinyaw on Thursday 25 February 2010 at 7:59 pm

Ubiquitous Broadband

As a luddite, I did not pay much attention to those 2G, 3G mumble jumble. I turn on my cell phone when I wake up. I would make several phone calls in a day and send/receive some text messages. That’s about it. For richer networking experiences and services, I fire up my trusted laptop, connect with a broadband services, and surf the net the old fashion way. I ignored all MMS (text messages with a video or picture) and never surfed the net with my cell phone. I shook my head, inside, on those iPhone junkies who are hopelessly addicted.

This nice shell shattered when a friend showed me her dongle a couple of years ago in Asia. With her 3G services (based on HSPA, offered universally to all subscribers in that small country she lives), she would remove the SIM card from her cell phone, insert it into this dongle, insert the dongle into her laptop USB port, and surf the net at the speed comparable to a typical WiFi (slightly better than 5Mbs).

There used to be three ways to connect to the Net: fixed line, WiFi, and dial-up. Now we have the fourth: 3G wireless. Operators around world have pretty much all converted to 3G and are now gearing up on 4G deployment with a technology called LTE. Simply put, 4G network allow one to surf the net at two to ten times faster than a typical WiFi hotspot today.

This means, with that magic dongle, I would no long need to hunt for hotspots. I would have broadband wherever, via a fundamental telephony technology. What does that mean if I would skype through this service? Do I pay skype rate (that is usually free) or a voice rate (that is, currently, expensive for international calls). My guess is that operators would simply offer flat rate for global voice services, since distance or geo-political boundaries are no longer a factor in costs.

Why would I need that home wireless router if everyone in the family can simply connect whenever and wherever? To avoid everyone suffling their SIM cards between their cell phone and computer, I would like a box that accept a single SIM card and essentially NAT (Network Address Translation, a way to allow multiple computers to connect to outside via a single point) between those computers at home. This box is like a FemtoCell, with a small twist in access control.

This luddite still insists using his phone only for voice and text. I would still carry a laptop that would blue-tooth to my cell phone when I an on the road. Otherwise, I would connect via either a company connection or my home box. Since my laptop connection would be ubiquiteous, I would store my data on the Internet (in one of those clouds) so that it would be light and cheap. The laptop and the cell phone are essentially the same, only the former has a better keyboard, mouse, and screen.

After so many decades, 4G wireless finally would solve the “last mile” problem. Funny, it is not Fibre to the Home (FTTH) as so many pundits predicted. It would be LTE. I bet whoever coined that name is regreting greatly. Long Term Evolution? Come on!

Posted under Peek into my mind by sinyaw on Saturday 20 February 2010 at 7:45 pm

Next Page »