Extrapolation fail
I have a love-hate relationship with statistics. On one hand I was never particularly good at math. But I was always fascinated by statistics and all the easily accessible truth they provide. I always loved Googling some figures and making people shut up.
Of course then I grew up (which is a great thing, highly recommended to everyone out there), and I realized that statistics are more truthful than just making shit up, as people are always prone to, but they're not the whole truth. Actually statistics is what you do when you don't know the whole truth, and you need to take the long way to infer something about the situation. The point is that as useful as statistics can be, the whole truth is obtained by actually understanding the background and the mechanism of some phenomenon. Compared to that, stats is just cheating. Reading some figures is of course much easier than taking the trouble to understand the whole picture, and so stats are prone to produce hubris in the reader.
One effect of that hubris is the pervasive belief in trends. Somebody makes a graph with a line that goes up or down more or less linearly during a given period of time, and next thing you know there are tons of ink spilled writing about it, and trillions on government budgets assigned to deal with that inevitable trend. The best example of course is Global Warming. Somebody made a graph that said that From 1920 to 1980 temperatures globally were rising. Next thing you know the British Government is actually unable to deal with menial problems common to Egyptian and Chinese Civilization 4000 years ago, like floods, unless they can officially link it to global warming. No I'm not making this up.
Of course the truth is that the temperature trends do not continue forever.
I am not interested in entering this debate about which I know little about, but I do happen to read things that may apply. On the Cambridge History of Ancient China, they point out how North China since the beginning of the Holocene well into historical time (the Han Dynasty, 200BC), North China was much warmer, full of subtropical flora and fauna, and tons of bones have been found of animals that are only found today well into the south. The point out that that's why North China was able to sustain such dense and wealthy neolithic cultures. Unlike today, in which North China is very cold.
This blind belief on climate trends comes from the fact that we just don't understand how climate actually works and what affects it. So we get some data, cherry pick some to make a graph, and then think we know something that we don't.
Another good example of blind belief on statistical trends is Moore's Law. Computing power is increasing like crazy. So we make a graph with transistor count or FLOPs or whatever. Yup, going up. Which means that given enough time, we'll have computer which are smarter than people, which means that a point will be reached where this smarter-than-humans will make ever better computers themselves, accelerating progress and unleashing the great Singularity where a god-like AI will rule over all humans!! And that AI will know about us in the past and punish according to our present deeds, so you better give away all your money to Shlomo so he can run a swingers club and screw your girlfriend!
No I'm not making that up either.
Of course all this crap comes from not understanding what actually drives Moore's Law. Transistors don't make themselves after all. They are developed through a quite complex process, which I don't presume to understand. Jim appears to understand it though, and he says Moore's Law is dying.
Rather than believing in Moore's Law, I prefer Stein's law. That which cannot go on forever, will stop.
Perhaps the most important trend for reactionaries of every sort is demographic trends. All people started as babies, so who is making the babies and how many of them is very important. I've written often on this blog about fertility and what drives it. I think I have a good hunch about it, but the fact is we don't know for sure. That doesn't stop people form looking at the fertility rates, drawing a graph and writing a whole book about it.
That's what David Goldman, with his stereotypical evil scholar face, did on writing about Islamic birth rates. Islam is collapsing!! He wrote. Birth rates are falling and will soon converge with those decadent white goyim and disappear! Then he looked at the sky and felt Yahveh's spirit comforting him.
Now while I am not driven by Jewish chauvinism like Mr. Goldman, the idea of the arab demographic pressure disappearing did appeal me as a European. The idea of Arabs overrunning Europe has made me lose more than one night's sleep. I thank Mr. Goldman for curing that ail from me.
What I don't appreciate is his misleading. One good thing about the internet is that you don't have to take people's word on things, you can actually go and look for the data yourself. And Wikipedia is full of very good demographic data on every country on earth.
Back on my last post on fertility there was a discussion on muslim birth rates, and the example of Iran came up. Iran is famous for its precipitous fertility decline, going from 6.5 in the 80s to below 2 in 2000. Check it out here. The discussion went on about the evil Cathedralist-satanic influence that made Iran women stop making babies for their husbands. At that moment I received an email by some Joshua Stern that linked to this article pointing out how it wasn't a spontaneous influx of evil leftism that cut Iran's fertility rate. It was a covert government operation of thousands of make-shift abortion wagons going around the country to make women stop having babies because the country was running out of water and food. Take a look at Google Maps satellite view and you'll find it hard to find some cultivated land in that desolate desert landscape we call Iran.
So yet again, understanding the actual mechanism beat handily a superficial look at statistical trends. By the way this courteous reader, Mr. Stern proceeded to delete his email address and disappear forever. Which is a pity as I was looking forward on more fruitful correspondence. Oh well.
Anyway this counter-climatic explanation about Iran's fertility rates made me think about the rest of the Arab world. What's driving their decreasing birthrate? And so I took a look at all the Wikipedia pages on fertility. And what I saw didn't look pretty.
https://www.cia.gov/library/publications/the-world-factbook/fields/2127.html
Saudi Arabia has seen a fertility crash, from 7 in the 1980s to 2.17 today.
Kuwait hit bottom at 2.24 in the 2000s but has since picked up and 2014 estimate is 2.54.
Iraq kept 5+ fertility up to recently but seems the occupation has done some good (!) and it's currently at 3.5.
The Emirates had gone sub-fertility for a while but it has now picked up and is now at 2.36.
Jordan has been declining steadily from 7 in the 1980s to 3.16 today.
Egypt has been declining steadily from 5 in the 1980s to 2.86 today. They've been unable to feed themselves for decades though.
Tunisia hit bottom at 2.02 in 2002, then picks up to 2.15 in 2011, CIA gives 2.00 for 2014.
Algeria was 4.5 in 1990, 2.81 in 2008, then picked up to 3.02 in 2012, CIA gives 2.78 for 2014.
Morocco was 5.5 in 1980, 2.19 in 2010, CIA gives 2.15 for 2014.
I don't know you guys, but what I'm seeing here is not collapse. What I'm seeing is a fertility correction given the massive pressures that overpopulation is having on what is very arid and unproductive land, but the populations are not declining, and if anything fertility is stabilizing above replacement rate. Given that none of these countries is seeing much economic growth or increased Westernization, further culture-induced fertility drops are unlikely, and if anything the next Islamic Revival movement might compel them to start a new Arab baby boom.
Meanwhile Germany has been losing native population for 42 years straight, at an increasing pace.
The Rothschild's paper, The Economist, never the pessimist, nevertheless have started to raise the alarm about global demographic trends. It turns out that black Africans are still pumping out as much babies as they can, not using all those condoms sent by the Melinda Gates Foundation nuggets. Cell phones and internet access are also having no effect.
What cannot go on forever must stop, and so African demographic expansion must stop. There's no food in the world for 2 billion Africans, let alone 3 or 4 billion as they would hypothetically produce at this rate by the end of the century. And so they will starve, one way or another. The glass-full backside of overpopulation, the "demographic dividend", produced a lot of wealth when it happened in Western Europe back in the days, or in China in the 1990s. But of course the demographic dividend is a statistical concept. Figures on a graph wasn't what made Europeans or Asians productive, it was the particular product of their own culture, their biology, the technology of the day, and only then, the economics of having a lot of young people around.
But of course nobody is interested on what's actually driving Africans to have 7 babies per women. As nobody is interested in what's actually driving Arabs to have on average less than 5, but never less than 2 children. As nobody really wants to know what's the deal with the climate. All we want is data to write a book, an article, or a government memo so I can sound smart, get a promotion or make some money.
34 comments
-
reply
The highlight of this post, for me, was your passive-aggressive mocking of Eliezer Yudkowsky's "polyamory". 10/10!
-
reply
It's somehow comforting that our priests are still telling tales of frost giants fighting fire giants, but we mere mortals are responsible for the conflict. A nice combo of Paganism and Christianity.
-
reply
Well most of the iranian population lives in the wet zones around mazandaran and near Tehran. Nobody actually lives in the desert except a handful of impoverished nomads.
-
reply
Not a lot of room for 77 million people, is it.
-
reply
It does fine. People can live in houses smaller than 2,800 square feet, you know.
-
reply
Well it seems they didn't do fine when everybody was busy aborting. It's not about physical room, it's about water and food.
-
reply
>> Water and food for future, GMO foods. More calories per acre. If not, natural selection. Which happened lots of times, and made us what we are. Not fun to watch though.
-
reply
What the hell are you talking about. The historical reality is that Iran had a choice and they sent abortion wagons to every village in the country. People don't wait for famines and water shortages these days.
-
reply
Yeah, but who do you think they force to abort the town doctor or the village idiot?
-
-
-
-
-
-
-
reply
Don't use the CIA's "factbook", it's fiction. Use census results.
-
reply
Where does one get those.
-
reply
Bullshit. Many country-level of statistics reported by the CIA factbook are redundantly available from the UN, World Bank, and IMF, and the CIA's numbers are very consistent with those other sources. E.G. list of countries by GDP (nominal). Are they all fiction too? Are fertility rate figures especially likely to be fiddled? But furthermore, if you have a good, reliable, and convincing source on why we should believe the numbers in the CIA factbook are fiction (say, by showing one clear and substantial inconsistency between a census-derived calculation and the CIA's figure), then what you are implying is that the CIA is content to risk getting caught in the act, ruin the factbook's reputation and fabricate fictional numbers which are easily contradicted by your good source, also risking the ire of Congress (many of the current members of which aren't exactly enamored of the Company who would like to embarrass them or the President). And the CIA's pressing imperative to risk this is ... what? Yes, government bureaus create silly statistics all the time, and fiddle with equations and names and definitions ('food insecurity' is the latest nonsense), and torture the numbers until they confess. But almost always this is about politically-hot-button and exploitable domestic numbers. No one in the administration can possibly care about what some bureaucrat at the agency reports about fertility numbers in Niger. This is the paradox / irony of the bureaucratic-managerial democracy state. The fairy tale narrative is that democracy allows for multiple kinds of external auditing and supervision, and that this keeps people honest. Nope. Most of what bureaucrats do in fashioning and implementing policy is ignored by everyone except a tiny handful of experts and interested parties - and the guys that put the numbers in the reports even more so. In my experience, the more ignored an area is (because politically insensitive), the more professional and honest the reporting-bureaucrat and the more reliable his figures. Ideologues aren't much attracted to areas without many ideologue-mischief-opportunities, so outside those areas you get mostly honest, efficient, and competent types just trying to do their job well. Hell, for the most obscure data, you used to see some poor GS-13's name, email, and phone number right next to the set of numbers for which he was responsible and he'd be happy to stand by those figures and defend them rigorously to anyone who called him, which no one ever did. But when a number becomes politically important and gets lots of attention from many sources, then you can always smell a rat, though you can almost never actually find him, because he's hidden under layers of committees and working groups and proprietary black-box models. Part of the problem with having all those 'auditors' is that they aren't the honest intermediaries they claim to be, and instead are just shilling for the administration when they defend the manipulations involved. The theoretical mechanism of 'oversight' is worthless when someone in corrupt league with the supervised can obtain and maintain a position of respected overseer and maintain at least as much status and partisan loyalty for their sophistic defense of the system as some other auditor that detracts it. An ancient problem with rhetoric is that it is easy to pass off lies when you frame them as being in fair and balanced debate with the truth - people say 'well, it's debatable', and then proceed to shrug their shoulders, and apply their biases, tribal loyalties and heuristic presumptions. There is also the related meta-problem of passing off a lie by denying the valid debatability of the truth by accusing its proponents of the afore-mentioned equivocation-framing attack. I used to think that to address this problem it would be a good general rule for no wonk to ever defend the government (as I am doing) - it should be able to defend itself. But alas it doesn't work that way. The government very rarely defends itself and indeed prohibits civil servants from defending it (and themselves) on their own (perhaps out of expectation that it simply won't be believed anyway), and instead uses these flack wonks and other media professionals to run interference for it (probably because the public invests more credibility in these 'independent' sources.) This is profoundly frustration when one finds themselves the innocent target of a smear-job, can easily prove the lies false, and yet finds that the department leadership will not defend itself. That's usually a good indication of FDR's old trick of the administration wanting to do something that is controversial, and needing to be 'pressured' into doing it from the outside so that they can fake a capitulation to the Potemkin rancor.
-
reply
This is great stuff and you should post it on your neglected blog and start a good discussion on the incentives on bureaucratic performance.
-
reply
In my defense, I've just returned from traveling with very limited internet for nearly three weeks.
-
reply
That's a lot of traveling.
-
reply
Over for a while, thankfully. Though I can strongly recommend Air New Zealand. Actually, I fell in love with that whole country - it's loaded with stunning natural beauty.
-
-
-
-
reply
You're being obnoxious about a claim you haven't even supported with information, not even someone's argument.
-
reply
A single example would do a lot for your credibility.
-
reply
Even if you factor in population momentum? A population with a low TFR can still grow quite a bit if the population is skewed towards young people with their reproductive years ahead of them.
-
reply
Indeed ; credible or not, 5371 isn't enhancing his credibility just at this moment. The pop of Saudia is very young: TFR was nosebleed-high there 20-25 years ago. (When they first got into oil, ~1950, the pop was 6M or something.) FWIW I have a vague recollection of CIA Factbook being nothing much. it is frequently used as a resource for academic research papers and news articles,\[5\] despite the dubious quality of its information (see below) I did not see below.
-
reply
I shouldn't chastise though ; I was a little off in my concept of TFR not many moons ago.
-
reply
ok, I saw below. It's not a knockout (parts are vague) but it's worth the 50 seconds of reading http://en.wikipedia.org/wiki/The\_World\_Factbook#Factual
-
reply
needless to say, cia is cryptofascist jackboots wall to wall while wikipedia and NPR simply present neutral information to benefit mankind regardless of race or ethny (except russians, who vivisect thousands of homosexuals per year), so there might be some friction between the two sides
-
-
-
-
reply
You're an ass. Out of here.
-
-
reply
I think Sailer remarked that he'd seen some hacks extrapolating about men's and women's 100m sprint world records in the 1960s (when men were stuck at the 10.0 second record while women had shaved a few tenths of a second off theirs during the decade) and how women would surpass men at some point. Might have been just one of his reductio ad absurdum -jokes, though.
-
reply
That Iranian fertility/ecology article is an interesting one, and it seems that 'family planning' policies in at least a semi-modernized society can generally only go one way. Money quotes: They were seeing modern women on television — including herself, as she and other gynecologists were now frequently on Iranian TV programs. “Families would find my telephone number. They’d ask, ‘How did you get this degree? How can we educate our daughters like you?’ ” Increasingly, the answer was easy. All the accolades that Iran’s family- planning program received in forthcoming years cited one indispensable factor: female education. Not just primary and secondary, but university. In 1975, barely a third of Iranian women could read. In 2012, more than 60 percent of Iranian university students were female. It's as if freeing women from the 'barefoot and pregnant' -script results in a slow, sclerotic demographical decline. One of those almost comical quirks of the human condition. Now, despite the Iranian high command's recent efforts trying to boost the birth rate back up, the people aren't biting any more. Singapore has a similar problem, as has been discussed here before. There seem to be very few carrots that can affect the fertility rate upwards once the correction has happened. It can't be done in subsistence villages in Iran, it can't be done in the jet set circles of Roppongi. Maybe both places are cramped in their own ways, and people are just reacting to their environment. It's also curious that Iraq has kept the regional high mark in fertility (after Afghanistan, presumably). Maybe an external threat or a conflict causes your womenfolk to feel the visceral need to pump up the numbers on your team (although a lot of your other data points are also somewhat fractional, at least internally). One thing that remains unknown is what exactly caused the the fertility correction in other Arab states. Did they also look the other way with national abortion programs, or are their trends comparable to what Iranians would have done as well, had they not warred with Iraq for years in the 80s?
-
reply
Well the Afghan example shows that treating women like livestock leads to very high fertility. Also there's double causation here. Giving women civil rights depresses fertility, but depressing fertility first (as in Iran) also results in giving women more power, as they find themselves with more free time to socialize and nag their husbands.
-
-
reply
[…] Fertility and extrapolation. […]
-
reply
I don't know anything about Arabic fertility, but those paragraphs about statistics are very good. The only thing that's missing is that statistics is often useful as a tool to generate ideas about underlying mechanisms, to check hypotheses about same and to find out unknown factors in your model of the mechanism. All physics students learn (or should learn) how to do this in simple cases, although in graduate physics and in less exact sciences the methods become much more difficult. As for trends, have you seen this of global sea ice coverage?
-
reply
The x-axis is a bit short for my taste.
-
-
reply
Don't forget that AI folks may or may not have actually shown that flops are intelligence. Moore's law describes a doubling of computing power, but if it's ultimately a dumb machine, even a machine a million times faster than what we have is just a very fast idiot. Judging by the comprehensive failures of AI research, it looks like nobody human is smart enough to make a truly thinking machine - all it can do is very quickly execute something a human already thought of.
So we get some data, cherry pick some to make a graph, and then think we know something that we don’t.
Accept your Ignorance! (Sorry, I may have a problem.)
-
reply
[…] “But of course nobody is interested on what’s actually driving Africans to have 7 babies per women. As nobody is interested in what’s actually driving Arabs to have on average less than 5, but never less than 2 children. As nobody really wants to know what’s the deal with the climate. All we want is data to write a book, an article, or a government memo so I can sound smart, get a promotion or make some money.“ […]
-
reply
[…] did write a post on being cautious about extrapolating trends, but you never […]
It really was. EY can't be mocked often enough.