You didn't expect the application you developed called "Skynet" that has an eye of sauron as it's logo to be used by the military? Sorry, I don't believe you.
I only made a prototype of the torment nexus as illustrative satire, I didn't think someone would actually ship it to production! All satire is dual use.
I would expect any invading forces would need to be surveilled by planes or satellites, or perhaps drones, since they would immediately destroy something like flock cameras.
I don't think the author is suggesting we shouldn't build useful things. He's saying to the people who built the military system that they shouldn't be doing that.
It's the last step that's the problem, not the people who built all the things they stand on. Otherwise we'd be suggesting that no one build a database engine or a digital camera or a computer ever again in case they're used to build evil tech.
I think there's a distinction between creating generic info-gathering technology that helps them gather their own data from their own users, and you gathering data that the government can then ask (force) you to hand over.
All technology can be weaponized. It can also be de-weaponized.
One of my favorite late-night thought experiments, is "what would it take to turn an AK47 or an AR15 into a water pump?" .. the possibilities are endless. "How could a tank be turned into a bicycle factory?", another great one worth the effort to think about.
The only difference between peaceful technology and a weapon is the intention of the user. The very real spiritual intention.
De-weaponization starts with the human being. Those who think weapons are necessary must always take lesser priority than those who invent technology to make the lives of their fellow human beings better. There are so many examples of the tool-makers winning harder and at greater scale than the bomb-manufacturer. We wouldn't be here now if the balance was the other way - our world is however threatened by having this scale weighed heavily, by those who think murder is the solution to anything.
It's a spirit thing. Not spiritu-al, a spirit thing. What is the spirit of the designer? If they want to help humanity, it will be difficult technology to weaponize (although not impossible). It will be easy to de-weaponize, also.
We need to focus more on the spirit of help, and less on hurt.
Imagine if those F35 mounting pylons were used to drop field-portable water-purification systems directly where they are needed most.
Imagine if they could be used to drop a copy of archive.org, anywhere in the world, in any language.
Imagine if those ships were being used to bring power to the villages that need it most. Instead of glide bombs, we have glide libraries. Instead of cluster munitions, we instead drop cluster medical kits!
Oh, the glory to be had by those who would do this - the allies in the poor and weak of the world to be found! Imagine this happening, changing the intention from hate and fear, to respect and love.
All it takes is the spirit, a bit of imagination, and any and all technology can be de-weaponized.
It is the warrior narcissist who stands in the way of this. Disarm them too. Drop books, not bombs!
> Imagine if those F35 mounting pylons were used to drop field-portable water-purification systems directly where they are needed most.
Except we can (and do!) accomplish that with much simpler airplanes. The staggering costs and complexities of an F35 (the thrust:weight, maneuverability, and stealth) don't make sense for any "peaceful" application.
The F35 could be used to mass-distribute field medical kits on Gaza today, and it would help literally thousands and thousands of fellow human beings in the action.
That US$2,000,000 bomb package could just as well be a high tech water purification system, deployed within hours directly at the location of the dying children, straight from the F35's wings, with haste and urgency! And it would go a lot, lot further towards peace-making than the current package of fertilizer being used to sever children's limbs.
The stealth - yes, we could throw that away immediately. Launch the solar-powered desalination package with great fanfare and aplomb! That is the point!
The stealth is necessary because the shame of spending a trillion dollars is high, and the fundamentalist warrior narcissist needs to hide, to justify their heinous acts, to conceal their identity which they fear yielding to any cause other than their own mirror.
Yes, de-waponize. Let me take the F35 for a ride to drop books on those cultures for which we find little compatibility - so far. I'll go back in a year and be sure there is peace. Maybe I'll walk then, easily, among the newly aligned. Those who gained understanding from the powerful machines in the sky, instead of death, mayhem and misery.
It might actually be wise [1] to create a new, slightly less permissioned license for open source wherein the open is defined as anything that isn’t in a kill or surveillance chain or, more broadly, for military use…
That wouldn't be open source. OSI includes the clause "No Discrimination Against Fields of Endeavor"[0] for its definition of open source. Famously, the JSON license[1] has a clause "The Software shall be used for Good, not Evil," so the Free Software Foundation considers it nonfree.[2]
Arguably, any such clauses - similar to attempt for "non commercial" licenses would do more harm than good.
If you actually wanted to respect such a license,you would have to endlessly argue what is dual use technology for example, the same as people argue if advertisements on project website or tangentially related donations make the thing "commercial"?
While at the same time, any "bad guys" will just ignore both.
The idea that a defense manufacturer would be stopped by this is too funny. The code won't be auditable by you so how will you know? Secondly even if you did find out the most you'd get is a payout. Thirdly if you believe that missile aren't running GNU/Linux I have a bridge to sell you.
I've contemplated writing an email to RMS asking him how he feels about the use of GNU software in military and surveillance systems and if he could do it over if he would do things differently to alllow users to put a clause in their OSS licenses that forbid such things.
Why do you want the people risking their lives to protect your freedom to have worse artillery? I don't understand this thinking at all. If my country is going to shoot something, I want them to be able to control what actually gets hit. I swear, people take borderline religious stances on things to the point that they no longer make sense.
The only thing that protects my freedom is active peace-making at levels greater than active war-making.
We are all under a great deal of threat precisely because borderline religious warrior narcissism has taken precedence over emphatic, unstoppable peace-making.
The USA has been bombing people into oblivion "protecting peace", for decades now. Is the world safer now?
No, it very definitely is not.
>Why do you want the people risking their lives ..
I want people risking their lives to make great peace. That is not happening at the scale that it should be.
Instead, the lazy, less noble warrior narcissist, has the world by the nose. The only difference between an utter criminal and a war fighter is a very, very thin line of paper which makes murder in one case illegal and in another case, legal. This paper is tattered and torn and covered in the muck of the victims, held high by the victors to justify their crime.
However, we see today the fallacy of this condition.
If your country is going to shoot something, I want to be damn sure it has done everything ethically, technologically, and administratively possible to have first made peace, at all costs.
That is simply not evident in any of the current world conflicts.
Instead of engaging in foreign cultures, we in the West march in lock-step behind those who have deemed foreign cultures inferior and worthy of nothing but destruction and despair.
I want to see the warrior narcissist who profit from this condition, disarmed and made irrelevant. The only way that happens is if human life takes precedence over human identity.
No humans identity is worth more than another humans life.
Risk your life for true peace this way: Before you murder, abandon yourself!
Many wars are simply what you’d expect from violent leaders who realize they can order other people into combat for them. But some wars really are for the greater good. There are definitely militaries that I want to see win, and militaries that I want to see lose.
I realize I don’t have any code that would be of any use to any military. If I did, I don’t see how I could write a license to allow the use of my code by only people I agree with.
I come from a military family - supporting soldiers and the policies of an administration are two different things. The commenter is correct in that our war in Iran has nothing to do with our freedom and everything about pushing the political agenda of a foreign country.
> Why do you want the people risking their lives to protect your freedom to have worse artillery?
Hi. Veteran here. I protected his freedoms to do just that. You asking this question in that way insults veterans, whether you intended it or not. People using this line of thinking usually only care about veterans when it can win them arguments, and do not care any other time.
> I don't understand this thinking at all
Then, instead of asking questions with inflammatory language, you should humble yourself and ask respectfully so you can understand.
> Why do you want the people risking their lives to protect your freedom to have worse artillery
If better artillery is being built on open source tech then everyone has better artillery. Not just the people "protecting you" but also the people trying to kill those people
I'm going to assume that you're an American, speaking from an Americentric point of view and I'd like to take this as an opportunity to discuss the fact that Hacker News is a website that is available to people around the world. Many of those people are not American citizens and do not have the same relationship with the US military and government that you do.
I doubt this is enforceable. Are you going to sue a government secret program because they used some open source code in a way that violates some weird clause?
Yes, citizens should sue the government when the government takes actions which are grounds for suit. The government sure will come after your money if you do things they say you shouldn't, and what's good for the goose is good for the gander.
You won't know, they'll say it's for national security so licenses don't matter, and then if you keep going they'll make sure your life is a living hell and your bank accounts closed.
I'm begging tech bros to understand that abusive governments _do not care about abusing you_
On the other hand, it may be worth publicly and unambiguously stating a position even if unenforceable. This seems to be what most modern protest movements are based on. For instance, how many software repos "stand with Ukraine", even though they won't make an impact on frontline military reality?
Ah yes, they're carpet bombing people and indiscriminately killing fighters, adults and children, but they're going to be stopped because... the software they use is two clause GPL.
The other rule here is that you cannot avoid politics if you want to create a better world. If you want the freedom to develop technology without worrying it will be used by a police state, you need to put in the work to make sure a police state doesn't happen - you cannot ignore politics and just "focus on technology."
On the other hand, a few heuristics will get you pretty far:
Don't build technology that interacts with people in ways they did't consent to. Don't build skinner boxes. Let the user decide who gets whatever data they generate while using the thing. Let the user control how it will work, insofar as it doesn't contradict the other heuristics.
These are political principles, but they're pretty basic. You don't have to keep up to date with the news, just check in with your morals once in a while.
Yeah, there’s definitely good design principles, but as others have pointed out, it’s extremely difficult to build technologies that are not trivially dual-use.
And, to be clear, I’m not saying “politics” as in “did you hear what Trump said?” I’m saying politics as in “what is your city council doing, how are your neighbors doing, and what are the consequences of your work and economic choices and who’s paying them.” Most of the news is a circus; that stuff is not.
Politics in the broadest sense is just any decision-making process where a single decision affects multiple people and those people don't all agree on which decision should be made.
How do you make sure a police state doesn't happen? Block traffic? Tweet? Cut off ties with friends and family that don't share your political beliefs?
Of course you can avoid politics. Just be a good person. Vote for who you think is the best candidate at the time. And move on with your life.
Ironically the people most involved in politics don't have their own house in order. They're often mentally ill, over educated, underemployed and without meaningful ties in their life (e.g. family, real life community, etc).
There are a lot of answers to this, but since we're likely coders in here, there's a software design answer worth pointing out.
Watch how police states behave. What tools do they use? Typically, it's roots of trust. They take down sites by tampering with their domain name. They MITM web traffic by compelling certificate authorities to do unstrustworthy things. They spy on cell phones by mimicking external authorities that the phones are wired to trust explicitly and without input from the user. They insert devices into ISP infrastructure which users tend to assume is neutral.
So if we want to prevent this: Don't tolerate roots of trust that the user cannot configure. Use a threat model that includes nation states. Don't design with the assumption that trusted third parties will remain trustworthy. Think about how to protect your users privacy. Prefer peer-to-peer where possible, because your server may at one point become compromised and harm the users. Think about how they can protect themselves if that happens. Give them control over whether to install updates. Publish your API so users can make unauthorized clients if they don't trust yours. Prefer repeatable builds, and check hashes to spot malware. Review contributions and be wary of binary blobs that may have hidden use cases.
Mostly this is just sound engineering. It means doing your job well, which is not always the same as doing the job the shareholders asked for. They invested so they could have leverage, so ask yourself ahead of time if those are the same kinds of levers that police states want to pull. Shoot them down in the design phase if possible, sabotage them after the fact if necessary.
By being durably involved in your community and by being aware of and not ignoring the consequences of your actions, of the policies you advocate for, the actions of the politicians you endorse, etc. By not just voting once and moving on.
What are you talking about? Should you hound co-workers that don't work for you? Leave pamphlets on cars with bad bumper stickers? Subscribe to the New Yorker?
If you're interested in politics you should read it, but you should not let it affect your life in any meaningful way.
You're being needlessly antagonistic and it makes me think you haven't actually explored local options. Let's start elsewhere: what sort of civic engagement are you involved with? Have you ever submitted a FOIA request? Been to a community advocacy group?
An objection of conscience is definitely a right you have as a professional, but it's a neutral action, you refuse to do something, and you let someone else do it. If you really are against it, you actively do something about it, like presenting a law or making a criminal complaint.
That said, they are both compatible, you can refuse to do something, and make efforts so that nobody does it.
But I think the the latter is the actual core security model of our society, and refusals aren't really effective to the point that I wouldn't worry much about it and just focus on compartimentalizing and executing on my duties. Isn't it theoretically possible that upper management divides duties so that engineers cannot accurately judge the ethical correctness of the task and the overall mission? As an individual you don't have the right tools to make such an evaluation. You can be pushed to investigate further, but the moment you do so, you cease your role as an engineer and put your citizen hat on.
Actually, you cannot avoid sex. Everything is about sex, even politics. If you want the freedom to develop technology without worrying it will be used by a police state, you cannot ignore sex and just "focus on technology"
There's a deep failed idea here. It's the idea that technology is an alternative to politics, period. I think that whole idea has failed.
You can't avoid building technologies that could be used by a police state, because literally everything can be used for warfare or aggression. The first person who sharpened a stick or a rock created a weapon. Everything is a weapon. Everything is dual-use.
If you want to prevent a police state or any other undesirable political outcome, you have to confront the politics directly. "Change the tools and you change the rules," an old saying from 1990s cyber-culture, is an idea that I think has failed. Instead what happens is that the rules determine what's done with the tools. Those 90s cyber-hippie and cyber-libertarian types vastly underestimated the power of the political to the point of making a category error.
Cryptocurrency is a great case in point. It started as an attempt to create a technology that would act as an escape hatch from corrupt casino capitalism and state-backed financialization. Instead it became an insane absurdist parody of those things. The same types of corrupt individuals who profited off "too big to fail" and the rest of it simply took cryptocurrency and used it to implement the same type of system but on a block chain. The rules (and rulers) changed the tools, not the other way around.
Not everything is dual use. Some information technology grossly favors centralization or decentralization of power based on the type of information problem it solves.
Cryptocurrency is clearly a counterexample to your main point. The fact that some people use cryptocurrency tech to implement centrally managed private currencies is largely irrelevant. To the extent that cryptocurrencies are used to implement private currencies, it largely just hampers them technolgicially if it has any effect at all. These projects just like to use the term cryptocurrency to imply some level of legitimate decentralization or security that doesn't really exist.
It's like saying that because you can use a screwdriver as a weapon, it's inherently a destructive technology (or some mixture of destructive and constructive). On closer inspection it's clear that the screwdriver doesn't sit on the pareto front for weapons, but it does sit on the pareto front for manufacturing tools. So a screwdriver is clearly a technology whose existence favors construction over destruction. Similarly I would argue that technology like bittorrent or bitcoin favor decentralization over centralization.
As another example, surveilance technology favors centralization. None of what we are seeing now with flock would be possible without cheap mass produced digital cameras. The balance of technology favors cheap cameras (and counter-surveilance is often impossible or expensive) and it favors centralization because you need a lot of data to create inferences based on this spurrious data.
If we could have predicted these effects, perhaps we could have delayed the development of cheap cameras, but it just goes to show how complicated this game is. We could just as easily have said that more cameras would lead to unprecedented public transparency (e.g. bodycams on cops, dash cams). Unintended consequences.
I mean you kind of can. The whole premise of AI is basically that we'll make white collar administrative compliance work dirt cheap in the long run and we'll all be richer for it and we'll sidestep the whole "fight with all those industries" step because the sheer economics of it will make it happen and as those industries become cheaper/poorer there will be even less reason to not kick them to the curb. Kind of like how once certain state shed a lot of their manufacturing or agriculture, adopting policy that's bad for the rest became easier. Or how now that solar is cheap we mostly DGAF about coal.
The Marxists were wrong about the inevitability of history no matter which direction or by what mechanism one wants to assert it. You’ve gotta put in the work no matter what, there’s no eliding the actual effort of politics.
The "work" here is changing the technological picture to the point where the political one becomes a non issue, at least for whatever incremental step you're looking at. Because once the technological picture changes enough, whoever your enemy is has to essentially roll over and take it, or your society will get out competed by some other one that did adopt it.
Like no amount of politics is putting the allure of the wealth potential of using a printing press to disseminate information cheaply back in the bag even if that comes at the expense of ending a gravy train for a subset of the current winners in your society.
Recently my wife got locked out of her bank account - all cards blocked, unable to log in to the website or app, so effectively her whole finances were frozen - because she was trying to order a new bank card over the phone and some system couldnt do voice recognition (which we did not know they even used). We were getting ready to get on a plane to NYC, from California (she banks with TD bank) but after a tearful session of phoning anyone and everyone we could at TD, suddenly it magically got reversed.
The problem is not just the tech, its building systems where humans are essentially slaves to the system, with no recourse or power over whats happening.
Few things are more important in modern life than access to your money. That's why I would never use an online bank. I use a local credit union but "local" with actual branch offices you can walk into and talk to a human are the important thing.
I tend to think the same way, but I've personally had far more issues with my local (largest in the region) credit union. Their (human) customer support reps frequently make mistakes processing my share certificates that automation would have otherwise avoided.
TD bank has branches but that's not super helpful if you're at the airport. the helpful thing is having a "relationship manager", someone you can call or email directly - you can get that at a big bank or a small one, and you don't need tons of assets, a lot of banks its having your direct deposit with them is enough to qualify for some special relationship.
I'm in the UK and it always seems weird to me how in the USA its totally normal to publically register yourself as either a Democrat or a Republican. To me seems like the kind of thing that should be private. In the UK I might choose to be a member of a political party but no-one needs to know that except that particular party. Or I can just join no party and vote in private on polling day.
It seems quite weird. Our parties are generally quite weird because were are not supposed to exist in the first place and so most of the rules are sort of ad-hoc.
I don’t belong to one, but it is sort of annoying to not be able to vote in the “primary elections” (where the parties pick their candidates). My state has a strong partisan lean so this is the only election that actually matters here. Oh well.
You really think taking an ethics COURSE will do anything?
How are you supposed to 'learn' ethics?
Never understood the point of those. My university had over 4 for the cyber students and we all went on to work for the MIC contributing to blowing up brown children in the middle east anyway.
Clearly, our problem must have been that we didn't have 5 mandatory time-waster ethics courses. That surely would have solved the matter. Add in a few extra "racism is bad" and "don't be a meanie" courses to round out the collection, just for good measure.
Every business school and MBA program has ethics course requirements. Does it really make a difference? Maybe it does, and things would be even worse than they are otherwise. But I'm not sure.
Especially if it has concrete historical examples of technology intended for good and how it was later used for evil, rather than something more generally-philosophical.
The automobile, the transistor, the Internet, and basically every other technology ~could~ be used by a police state. I'm not sure how that is a helpful rule at all.
The article says the same. I think the point is to be aware and try to build things in a way to minimize the chance for abuse. There's nuance and the author says as much.
This is missing the "(2013)" at the end of the HN link, but it does seem weirdly prescient as we say a rise of more polarised politics across the World, just 13 years later.
Clearly you need to consider how general the capabilities and use cases of the technology is. The most general-purpose technologies will always have obvious use cases for police states and other bad groups or people.
Defaulting to discussing the US here: let's start with reducing the state's size - it shouldn't be the world's largest employer, it shouldn't be able to run an unfathomably large deficit, it shouldn't be able to declare war without congressional approval. Technology to run a police state is just the last step in a long line of ridiculously unacceptable steps.
Let's slip in your pet bugaboo. Workforce size very little to do with state surveillance. In fact, mandating that it shrink can result in blowback, just like we have seen with how the US provides a great antagonist for China to evolve its own capabilities faster than if we had done nothing.
How does the number of state employees de facto result in a police state?
I agree with you on our obscene debt and our obsession with "conflicts" around the world, but I'd argue we're seeing the mass reduction of state employees and it's just making everything worse (eg massive deregulation and the destruction of checks and balances).
It won’t work for any country. They all want more control, and also create a class of people depending on government money to be on their side. Only people losing out are the powerless ordinary citizens and taxpayers
The sad irony of "Share this post on..." with neat list of all the services doing exactly what the post warns against. They can and often are already used by police state.
I've been arguing with folks on Reddit about this lately when it comes to Google's changes to Android (where they've made it more difficult to install any software on your device without Google first tracking the identity of the developer). My argument ultimately comes down to the fact that Google (and Apple already) have built a technology that gives them immense power to control individual computing freedoms, and no matter their internal policy, they still must follow US law. No amount of promises or legit good faith on their part matter, they've made a dangerous technology and are behold to a government that is very eager to censor anything it doesn't like. It's so disappointing to see 1. Google engineers build such a thing, and 2. For people to support it.
Civic Hygiene is the opposite of how you were taught to write code...
>It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter.
Has this philosophical approach ever worked? It's a commonplace in the hard sciences and engineering fields (ie, refusing to work on weapons systems), but weapons systems are elite and well-compensated engineering projects that don't have any trouble finding talent. The same thing goes for NSA, which has a line out the door from elite academia.
I'm not dismissing the sentiment here. It's morally sound. You should act in accordance with your principles and it useful to be reminded how those principles intersect the world. I have similar guardrails (probably not the same as the author's). I just wouldn't want people to overestimate what can be accomplished with this approach.
A significant number of people in positions of power in the AI industry get their moral framework from science fiction books and a particular Harry Potter fanfiction. It’s hard to predict what ideas and philosophies will become important.
"Harry Potter and the Methods of Rationality" written by Eliezer Yudkowsky has become something akin to a sacred text among the rationalist community.
edit: posted is a podcast by Behind the Bastards about the Zizians, a death cult that branched out of rationalist community and which was responsible for six deaths, but it covers the rationalist community and the Harry Potter fanfiction as well.
I find it frusrating how the conversation around this almost always focuses on one side of the equation. Why is it always about preventing tools from being built in case authoritarians comes to power rather than preventing those authoritarians from coming to power in the first place? I'm far more scared off the authoritarian aspect than the tech. The Nazis didn't need Flock cameras to kill millions. It seems like focusing on the actual bad guys is going to be more effective because if the state wants to bear its weight on you, it rarely requires tech to do it.
The Nazis relied heavily on IBM technology for national census data (to identify jews), military logistics, ghetto statistics, train traffic management, and concentration camp capacity
I fully agree with your base perspective, that the main focus should be on preventing authoritarians from coming to power in the first place.
I also agree with your final point, that the government doesn't need tech to lean on a person.
We disagree in the middle, on how tech that makes evil orders of magnitude easier might be a contributing factor and might enable the government to lean on many more people than it would otherwise.
People have incentives to become more powerful - through wealth or in other ways.
So unfortunately, authoritarianism tends to have gravity going for it.
>The Nazis didn't need Flock cameras to kill millions.
No, but they were extremely happy to hop over to France and realize that we kept lists that indirectly listed jews. Actions have consequences that we may not be able to predict.
I remember an old internet post I made in the '90s saying that the internet was like Woody Guthrie's guitar, which had a label on it that read "This machine kills fascists". Where do I go to get my innocence back?
If you build a technology that improves human communication in any way, if successful it will likely be leveraged by a police state.
Libertarianism has its moments but it generally hasn't caught on because state capacity is unique and useful. And virtually every last bit of it can be used to facilitate a police state because it's... a state.
Sure, we could all live without IDs and use motio belli for defence but this arrangement will be soon replaced by someone more organized because there is a huge return to that organization despite its risks.
The article is not completely wrong in its premises, but phrased incorrectly.
This is not about technologies (after all, an authoritarian government will use many existing technologies, from the car to the computer), but about powers granted to a government, that could be used differently by the next party/majority in power.
The answer to that is to not grant government too much power in the first place.
Of course, this is often a contentious point for leftists, since they do want to grant government a lot of power in the first place; they're just concerned that the exact same powers will then be used by a different political side in a way that bothers a different category of people.
You will get downvoted to oblivion for even suggesting it here but it's so obviously a take that maps on cleanly to reality. Massive power centralization in government makes everything dual use in the service of a police state. The easiest thing to do is resist massive centralization of power in a government.
you get downvoted because to most people we don't care if its tyranny.gov or tyranny.com. "don't give powers to the government" just means they buy them from corporations. see: cellphone location data.
the solution is not to have the power exist at all, anywhere.
"Civic hygiene isn't about saying we distrust our current government - it's about not trusting the next government." I've made similar points. Granting one government administration more power, is by extension granting it to all future administrations. Do so only with great care.
Like others have said, I don't think it's possible to create technology that cannot be weaponized. I think it's possible (mostly) to design technology so that abuse requires more people, requires the leaving of evidence / paper trail, needs checks, etc.
But still, it's not possible to prevent abuse altogether and sometimes we even applaud breaking the rules (the movie Hunt for Red October for example kicks off with the Soviet captain jumping the guardrails).
Also, the most "benign" public infrastructure is usually the most impactful, e.g., water systems and electrical systems. Don't like the people in that one town who keep protesting and voting against you? Turn off their power, water, and also make it so that their trash doesn't get picked up for a year. See how they like it.
this is an important topic, and i hate that people forget how difficult it is. in any political conversation, it's pretty easy to pull this thread to unravel anyone's adamant belief in one thing or another (e.g. gun control). my biggest concern is that people seem less willing than before to actually have those discussions.. they'll immediately just label you as being opposed to their views, "against" them.. when really we need these conversations to find viable policy solutions that wont easily be used in the exact opposite way that they were intended when implemented
Technology is force multiplier. Everything could be used for oppression. Deal with it - the future is boot stomping on human face forever. There are lucky few that could choose whether they will be the face or foot. But it is inevitable.
Or we could stop all tech development. But it is not going to happen.
Edit; wrote this comment before I realised this is a blog post from 2013. In 2013 I would've agreed 110% with this.
-----------
This is one of those things that make for great post title and gets you all riled up at first but when you think about it, it's really shallow.
Most civic databases can be abused by any sufficiently authoritarian government. A homeless shelter's records may contain exactly the sort of information that could be used to get "undesirables" off the street. But those same records are useful for coordinating beds, outreach, medical care, etc. Surely the conclusion isn't that the shelter shouldn't keep records.
It seems much more useful to focus on limiting what gets collected, who gets access to it, and how long it can be retained. In other words, boring things like data-minimisation and retention laws. Somewhat amusingly, that gets you pretty close to the sort of principles behind GDPR, which isn't exactly universally loved around here.
I don't really disagree with the blog post's moral intuition. What bothers me is the emphasis on individual moral choices by the people building these systems. I've seen plenty of my peers happily work for companies that have spent the last twenty years building exactly this kind of technology, there will always be someone willing to build it. That's not an argument for abandoning personal ethics, but it is an argument against treating personal ethics as any form of serious societal safeguard. "good engineers should refuse to build them" means nothing.
Companies that agree to work on Chat Control deserve a special place in hell. One company that was eager to participate, hoping this harmful law would pass, is Atos. Name and shame them.
Assholes like this were all for the police state back when it was up to stop-the-spread “contact tracing”, real-time “fact-checking” my private messages, arresting people for facebook posts, and transing your kids.
Now that the red pedophiles are in charge instead of the blue ones, he changes his tune.
I think this runs into a basic issue that all large software systems that enforce the law contribute to locking down a society because laws tend to need be broken to be changed.
That a system can be avoided offends the investor who want their products to be ubiquitous.
The system that is not perfectly enforced offends the engineer who dislikes broken things.
Yet both are required if we as a society are to grow. I think the thing that it might be good to learn is when to say "I've done/had enough."
I think I could collect enough weight of books on "civil disobedience" at my local library that I could crush you underneath them, even if you didn't bother reading them.
the problem is Capitalism does not need Democracy to function or survive
many countries like Singapore, etc. prove this easily
Authoritarianism can even be desired by Capitalists if they know who to bribe to remove regulations and get approvals
and no surprise, that's exactly where the United States is right now, teetering on the edge of that hole, about to fall in
"AI" is going to be weaponized, already is, and any resistance against its abuse is simply overriden with threats to destroy profits or even company survival at all
There is some sort of "nerd naivete" that they are the ones with power to control those things.
If you're a nerd developing open source (not an insult; I'm in that category), our job is to develop effective technology.
If you want to do politics, go into law and/or politics (or into business, so you can fund/control politics).
>"But software which OpenBSD uses and redistributes must be free to all (be they people or companies), for any purpose they wish to use it, including modification, use, peeing on, or even integration into baby mulching machines or atomic bombs to be dropped on Australia. " -- Theo De Raadt
> There will always be someone willing to build that thing
… with sufficient time and money. As opposed to the minimal time and money required if the baddies are just pulling an existing solution off the shelf.
I hope you are joking, or haven't thought this through! If you sincerely believe and act on what you're saying here, you should be shunned from polite society.
Just because someone is willing to build it doesn't mean it has to be you.
We may all play some part in societies that have some degree of monstrous outcomes, but I feel like staying on the "tragic civilian casualty" side versus the "legitimate target" side of a freedom fighter's bombing campaign is probably the way to go.
It's not about someone willing or not, it's about you, an ostensibly grown adult who should think in terms of your own responsibilities instead of others'.
Terrible argument. You can say ”why not work as a concentration camp guard? There will always be someone willing to do it so you might as well cash in”
People cite Flock as an example of this. Yet the same people walk around with iPhones and Android devices that can record conversations (who talks to whom at what times and for how long), know our exact locations and log everything we do online.
If I had told you 30 years ago that we are going to build a surveillance network and make the people we are tracking pay for it all you would have laughed me out of the room. But that is what we have built. The modern Smart Phone.
Now we use our iPhones to complain about cameras on public roads as if they are watching us.
Very flawed argument, anyone can go for a walk without their phone on them and have a conversation, nobody would ask them why they don't have their phone on them.
Ostensibly they need a warrant to get that data from Apple/Google whereas they have a direct line to Flock data cause that's how that relationship works
Harder than it looks. Everything is dual use. Certainly didn't expect my game development telemetry[1] system used to improve artillery [2]...
[1] https://gdcvault.com/play/1012227/Development-Telemetry-in-V...
[2] https://www.army.mil/article/85934/solution_to_weapon_develo...
You didn't expect the application you developed called "Skynet" that has an eye of sauron as it's logo to be used by the military? Sorry, I don't believe you.
I only made a prototype of the torment nexus as illustrative satire, I didn't think someone would actually ship it to production! All satire is dual use.
There should be a distinction between helping the military and helping a police state.
Eg. Flock cameras don’t help the military but do help the police state.
Military tech often flows to cops.
In case of ground invasion (or just to detect enemy drones in the sky) having access tonlots of cameras is of military use.
Not a likely scenario, FWIW: https://www.military.com/history/5-reasons-why-geography-ame...
I would expect any invading forces would need to be surveilled by planes or satellites, or perhaps drones, since they would immediately destroy something like flock cameras.
I think you are just playing devil’s advocate.
No, it becomes a real problem when wannabe dictators start using the military for illegal policing operations:
https://www.brennancenter.org/our-work/analysis-opinion/cour...
I don't think the author is suggesting we shouldn't build useful things. He's saying to the people who built the military system that they shouldn't be doing that.
It's the last step that's the problem, not the people who built all the things they stand on. Otherwise we'd be suggesting that no one build a database engine or a digital camera or a computer ever again in case they're used to build evil tech.
Not sure about digital cameras or database engines, but computers and the Internet were, in large part, created for military needs first and foremost.
I think there's a distinction between creating generic info-gathering technology that helps them gather their own data from their own users, and you gathering data that the government can then ask (force) you to hand over.
>Harder than it looks.
But maybe easier than it sounds.
All technology can be weaponized. It can also be de-weaponized.
One of my favorite late-night thought experiments, is "what would it take to turn an AK47 or an AR15 into a water pump?" .. the possibilities are endless. "How could a tank be turned into a bicycle factory?", another great one worth the effort to think about.
The only difference between peaceful technology and a weapon is the intention of the user. The very real spiritual intention.
De-weaponization starts with the human being. Those who think weapons are necessary must always take lesser priority than those who invent technology to make the lives of their fellow human beings better. There are so many examples of the tool-makers winning harder and at greater scale than the bomb-manufacturer. We wouldn't be here now if the balance was the other way - our world is however threatened by having this scale weighed heavily, by those who think murder is the solution to anything.
It's a spirit thing. Not spiritu-al, a spirit thing. What is the spirit of the designer? If they want to help humanity, it will be difficult technology to weaponize (although not impossible). It will be easy to de-weaponize, also.
We need to focus more on the spirit of help, and less on hurt.
Imagine if those F35 mounting pylons were used to drop field-portable water-purification systems directly where they are needed most.
Imagine if they could be used to drop a copy of archive.org, anywhere in the world, in any language.
Imagine if those ships were being used to bring power to the villages that need it most. Instead of glide bombs, we have glide libraries. Instead of cluster munitions, we instead drop cluster medical kits!
Oh, the glory to be had by those who would do this - the allies in the poor and weak of the world to be found! Imagine this happening, changing the intention from hate and fear, to respect and love.
All it takes is the spirit, a bit of imagination, and any and all technology can be de-weaponized.
It is the warrior narcissist who stands in the way of this. Disarm them too. Drop books, not bombs!
> Imagine if those F35 mounting pylons were used to drop field-portable water-purification systems directly where they are needed most.
Except we can (and do!) accomplish that with much simpler airplanes. The staggering costs and complexities of an F35 (the thrust:weight, maneuverability, and stealth) don't make sense for any "peaceful" application.
The F35 could be used to mass-distribute field medical kits on Gaza today, and it would help literally thousands and thousands of fellow human beings in the action.
That US$2,000,000 bomb package could just as well be a high tech water purification system, deployed within hours directly at the location of the dying children, straight from the F35's wings, with haste and urgency! And it would go a lot, lot further towards peace-making than the current package of fertilizer being used to sever children's limbs.
The stealth - yes, we could throw that away immediately. Launch the solar-powered desalination package with great fanfare and aplomb! That is the point!
The stealth is necessary because the shame of spending a trillion dollars is high, and the fundamentalist warrior narcissist needs to hide, to justify their heinous acts, to conceal their identity which they fear yielding to any cause other than their own mirror.
Yes, de-waponize. Let me take the F35 for a ride to drop books on those cultures for which we find little compatibility - so far. I'll go back in a year and be sure there is peace. Maybe I'll walk then, easily, among the newly aligned. Those who gained understanding from the powerful machines in the sky, instead of death, mayhem and misery.
I feel really bad for you.
It might actually be wise [1] to create a new, slightly less permissioned license for open source wherein the open is defined as anything that isn’t in a kill or surveillance chain or, more broadly, for military use…
[1] for the community
That wouldn't be open source. OSI includes the clause "No Discrimination Against Fields of Endeavor"[0] for its definition of open source. Famously, the JSON license[1] has a clause "The Software shall be used for Good, not Evil," so the Free Software Foundation considers it nonfree.[2]
[0]: https://opensource.org/osd
[1]: https://www.json.org/license.html
[2]: https://www.gnu.org/licenses/license-list.en.html#JSON
Arguably, any such clauses - similar to attempt for "non commercial" licenses would do more harm than good.
If you actually wanted to respect such a license,you would have to endlessly argue what is dual use technology for example, the same as people argue if advertisements on project website or tangentially related donations make the thing "commercial"?
While at the same time, any "bad guys" will just ignore both.
The idea that a defense manufacturer would be stopped by this is too funny. The code won't be auditable by you so how will you know? Secondly even if you did find out the most you'd get is a payout. Thirdly if you believe that missile aren't running GNU/Linux I have a bridge to sell you.
I've contemplated writing an email to RMS asking him how he feels about the use of GNU software in military and surveillance systems and if he could do it over if he would do things differently to alllow users to put a clause in their OSS licenses that forbid such things.
IIRC that has been discussed many times & can be definitely found in mailing list archives.
He's already written that essay. It's more rhetoric than substance, but his position is clear. https://www.gnu.org/philosophy/programs-must-not-limit-freed...
Why do you want the people risking their lives to protect your freedom to have worse artillery? I don't understand this thinking at all. If my country is going to shoot something, I want them to be able to control what actually gets hit. I swear, people take borderline religious stances on things to the point that they no longer make sense.
"protect your freedom" does a lot of work here..
>borderline religious stances on things
+
>risking their lives to protect your freedom
=
nonsense.
The only thing that protects my freedom is active peace-making at levels greater than active war-making.
We are all under a great deal of threat precisely because borderline religious warrior narcissism has taken precedence over emphatic, unstoppable peace-making.
The USA has been bombing people into oblivion "protecting peace", for decades now. Is the world safer now?
No, it very definitely is not.
>Why do you want the people risking their lives ..
I want people risking their lives to make great peace. That is not happening at the scale that it should be.
Instead, the lazy, less noble warrior narcissist, has the world by the nose. The only difference between an utter criminal and a war fighter is a very, very thin line of paper which makes murder in one case illegal and in another case, legal. This paper is tattered and torn and covered in the muck of the victims, held high by the victors to justify their crime.
However, we see today the fallacy of this condition.
If your country is going to shoot something, I want to be damn sure it has done everything ethically, technologically, and administratively possible to have first made peace, at all costs.
That is simply not evident in any of the current world conflicts.
Instead of engaging in foreign cultures, we in the West march in lock-step behind those who have deemed foreign cultures inferior and worthy of nothing but destruction and despair.
I want to see the warrior narcissist who profit from this condition, disarmed and made irrelevant. The only way that happens is if human life takes precedence over human identity.
No humans identity is worth more than another humans life.
Risk your life for true peace this way: Before you murder, abandon yourself!
Where has my “freedom” been protected in the last ~40 years?
How do you know for sure that only your team is using the code? Are you ok with "the enemy" using your code to kill your soldiers, or you?
Many wars are simply what you’d expect from violent leaders who realize they can order other people into combat for them. But some wars really are for the greater good. There are definitely militaries that I want to see win, and militaries that I want to see lose.
I realize I don’t have any code that would be of any use to any military. If I did, I don’t see how I could write a license to allow the use of my code by only people I agree with.
because my country isn't protecting my freedom, it's killing schoolgirls in a country we had a perfectly fine agreement with four years ago
This level of ingratitude for people sacrificing everything for you is awful.
> because my country isn't protecting my freedom
is a fucking batshit thing to say. As you would find out very quickly if they stopped.
[delayed]
I come from a military family - supporting soldiers and the policies of an administration are two different things. The commenter is correct in that our war in Iran has nothing to do with our freedom and everything about pushing the political agenda of a foreign country.
> Why do you want the people risking their lives to protect your freedom to have worse artillery?
Hi. Veteran here. I protected his freedoms to do just that. You asking this question in that way insults veterans, whether you intended it or not. People using this line of thinking usually only care about veterans when it can win them arguments, and do not care any other time.
> I don't understand this thinking at all
Then, instead of asking questions with inflammatory language, you should humble yourself and ask respectfully so you can understand.
If they can't do their job non-violently then I don't want them to do that job.
> Why do you want the people risking their lives to protect your freedom to have worse artillery
If better artillery is being built on open source tech then everyone has better artillery. Not just the people "protecting you" but also the people trying to kill those people
I'm going to assume that you're an American, speaking from an Americentric point of view and I'd like to take this as an opportunity to discuss the fact that Hacker News is a website that is available to people around the world. Many of those people are not American citizens and do not have the same relationship with the US military and government that you do.
The link given was to a blog post from the US army. Good try though.
https://firstdonoharm.dev/
I doubt this is enforceable. Are you going to sue a government secret program because they used some open source code in a way that violates some weird clause?
Yes, citizens should sue the government when the government takes actions which are grounds for suit. The government sure will come after your money if you do things they say you shouldn't, and what's good for the goose is good for the gander.
You won't know, they'll say it's for national security so licenses don't matter, and then if you keep going they'll make sure your life is a living hell and your bank accounts closed.
I'm begging tech bros to understand that abusive governments _do not care about abusing you_
That doesn't mean you should spread your legs for them. This defeatist attitude is suicidal
No, treating your illusion of control as a religion is suicidal. You need a theory of power other than complaining to the manager.
https://en.wikipedia.org/wiki/Illusion_of_control
On the other hand, it may be worth publicly and unambiguously stating a position even if unenforceable. This seems to be what most modern protest movements are based on. For instance, how many software repos "stand with Ukraine", even though they won't make an impact on frontline military reality?
Ah yes, they're carpet bombing people and indiscriminately killing fighters, adults and children, but they're going to be stopped because... the software they use is two clause GPL.
You could sue cloud providers who are using your software and selling to the military
The other rule here is that you cannot avoid politics if you want to create a better world. If you want the freedom to develop technology without worrying it will be used by a police state, you need to put in the work to make sure a police state doesn't happen - you cannot ignore politics and just "focus on technology."
On the other hand, a few heuristics will get you pretty far:
Don't build technology that interacts with people in ways they did't consent to. Don't build skinner boxes. Let the user decide who gets whatever data they generate while using the thing. Let the user control how it will work, insofar as it doesn't contradict the other heuristics.
These are political principles, but they're pretty basic. You don't have to keep up to date with the news, just check in with your morals once in a while.
Yeah, there’s definitely good design principles, but as others have pointed out, it’s extremely difficult to build technologies that are not trivially dual-use.
And, to be clear, I’m not saying “politics” as in “did you hear what Trump said?” I’m saying politics as in “what is your city council doing, how are your neighbors doing, and what are the consequences of your work and economic choices and who’s paying them.” Most of the news is a circus; that stuff is not.
Politics in the broadest sense is just any decision-making process where a single decision affects multiple people and those people don't all agree on which decision should be made.
How do you make sure a police state doesn't happen? Block traffic? Tweet? Cut off ties with friends and family that don't share your political beliefs?
Of course you can avoid politics. Just be a good person. Vote for who you think is the best candidate at the time. And move on with your life.
Ironically the people most involved in politics don't have their own house in order. They're often mentally ill, over educated, underemployed and without meaningful ties in their life (e.g. family, real life community, etc).
There are a lot of answers to this, but since we're likely coders in here, there's a software design answer worth pointing out.
Watch how police states behave. What tools do they use? Typically, it's roots of trust. They take down sites by tampering with their domain name. They MITM web traffic by compelling certificate authorities to do unstrustworthy things. They spy on cell phones by mimicking external authorities that the phones are wired to trust explicitly and without input from the user. They insert devices into ISP infrastructure which users tend to assume is neutral.
So if we want to prevent this: Don't tolerate roots of trust that the user cannot configure. Use a threat model that includes nation states. Don't design with the assumption that trusted third parties will remain trustworthy. Think about how to protect your users privacy. Prefer peer-to-peer where possible, because your server may at one point become compromised and harm the users. Think about how they can protect themselves if that happens. Give them control over whether to install updates. Publish your API so users can make unauthorized clients if they don't trust yours. Prefer repeatable builds, and check hashes to spot malware. Review contributions and be wary of binary blobs that may have hidden use cases.
Mostly this is just sound engineering. It means doing your job well, which is not always the same as doing the job the shareholders asked for. They invested so they could have leverage, so ask yourself ahead of time if those are the same kinds of levers that police states want to pull. Shoot them down in the design phase if possible, sabotage them after the fact if necessary.
> Vote for who you think is the best candidate at the time
That is definitionally not avoiding politics.
I think you misunderstand what politics is, at a fundamental level.
By being durably involved in your community and by being aware of and not ignoring the consequences of your actions, of the policies you advocate for, the actions of the politicians you endorse, etc. By not just voting once and moving on.
What does this mean? This is all vague.
"Being aware"
"advocate for"
What are you talking about? Should you hound co-workers that don't work for you? Leave pamphlets on cars with bad bumper stickers? Subscribe to the New Yorker?
If you're interested in politics you should read it, but you should not let it affect your life in any meaningful way.
You're being needlessly antagonistic and it makes me think you haven't actually explored local options. Let's start elsewhere: what sort of civic engagement are you involved with? Have you ever submitted a FOIA request? Been to a community advocacy group?
> you should not let it affect your life in any meaningful way.
Oh, I’ve got really bad news for you.
The people most involved in politics are mentally ill? Got it...
This definitely feels true, yeah.
>you cannot ignore politics and just "focus on technology."
Stallman was right again
https://www.azquotes.com/quote/755239
An objection of conscience is definitely a right you have as a professional, but it's a neutral action, you refuse to do something, and you let someone else do it. If you really are against it, you actively do something about it, like presenting a law or making a criminal complaint.
That said, they are both compatible, you can refuse to do something, and make efforts so that nobody does it.
But I think the the latter is the actual core security model of our society, and refusals aren't really effective to the point that I wouldn't worry much about it and just focus on compartimentalizing and executing on my duties. Isn't it theoretically possible that upper management divides duties so that engineers cannot accurately judge the ethical correctness of the task and the overall mission? As an individual you don't have the right tools to make such an evaluation. You can be pushed to investigate further, but the moment you do so, you cease your role as an engineer and put your citizen hat on.
Actually, you cannot avoid sex. Everything is about sex, even politics. If you want the freedom to develop technology without worrying it will be used by a police state, you cannot ignore sex and just "focus on technology"
There's a deep failed idea here. It's the idea that technology is an alternative to politics, period. I think that whole idea has failed.
You can't avoid building technologies that could be used by a police state, because literally everything can be used for warfare or aggression. The first person who sharpened a stick or a rock created a weapon. Everything is a weapon. Everything is dual-use.
If you want to prevent a police state or any other undesirable political outcome, you have to confront the politics directly. "Change the tools and you change the rules," an old saying from 1990s cyber-culture, is an idea that I think has failed. Instead what happens is that the rules determine what's done with the tools. Those 90s cyber-hippie and cyber-libertarian types vastly underestimated the power of the political to the point of making a category error.
Cryptocurrency is a great case in point. It started as an attempt to create a technology that would act as an escape hatch from corrupt casino capitalism and state-backed financialization. Instead it became an insane absurdist parody of those things. The same types of corrupt individuals who profited off "too big to fail" and the rest of it simply took cryptocurrency and used it to implement the same type of system but on a block chain. The rules (and rulers) changed the tools, not the other way around.
Not everything is dual use. Some information technology grossly favors centralization or decentralization of power based on the type of information problem it solves.
Cryptocurrency is clearly a counterexample to your main point. The fact that some people use cryptocurrency tech to implement centrally managed private currencies is largely irrelevant. To the extent that cryptocurrencies are used to implement private currencies, it largely just hampers them technolgicially if it has any effect at all. These projects just like to use the term cryptocurrency to imply some level of legitimate decentralization or security that doesn't really exist.
It's like saying that because you can use a screwdriver as a weapon, it's inherently a destructive technology (or some mixture of destructive and constructive). On closer inspection it's clear that the screwdriver doesn't sit on the pareto front for weapons, but it does sit on the pareto front for manufacturing tools. So a screwdriver is clearly a technology whose existence favors construction over destruction. Similarly I would argue that technology like bittorrent or bitcoin favor decentralization over centralization.
As another example, surveilance technology favors centralization. None of what we are seeing now with flock would be possible without cheap mass produced digital cameras. The balance of technology favors cheap cameras (and counter-surveilance is often impossible or expensive) and it favors centralization because you need a lot of data to create inferences based on this spurrious data.
If we could have predicted these effects, perhaps we could have delayed the development of cheap cameras, but it just goes to show how complicated this game is. We could just as easily have said that more cameras would lead to unprecedented public transparency (e.g. bodycams on cops, dash cams). Unintended consequences.
I mean you kind of can. The whole premise of AI is basically that we'll make white collar administrative compliance work dirt cheap in the long run and we'll all be richer for it and we'll sidestep the whole "fight with all those industries" step because the sheer economics of it will make it happen and as those industries become cheaper/poorer there will be even less reason to not kick them to the curb. Kind of like how once certain state shed a lot of their manufacturing or agriculture, adopting policy that's bad for the rest became easier. Or how now that solar is cheap we mostly DGAF about coal.
The Marxists were wrong about the inevitability of history no matter which direction or by what mechanism one wants to assert it. You’ve gotta put in the work no matter what, there’s no eliding the actual effort of politics.
The "work" here is changing the technological picture to the point where the political one becomes a non issue, at least for whatever incremental step you're looking at. Because once the technological picture changes enough, whoever your enemy is has to essentially roll over and take it, or your society will get out competed by some other one that did adopt it.
Like no amount of politics is putting the allure of the wealth potential of using a printing press to disseminate information cheaply back in the bag even if that comes at the expense of ending a gravy train for a subset of the current winners in your society.
Recently my wife got locked out of her bank account - all cards blocked, unable to log in to the website or app, so effectively her whole finances were frozen - because she was trying to order a new bank card over the phone and some system couldnt do voice recognition (which we did not know they even used). We were getting ready to get on a plane to NYC, from California (she banks with TD bank) but after a tearful session of phoning anyone and everyone we could at TD, suddenly it magically got reversed.
The problem is not just the tech, its building systems where humans are essentially slaves to the system, with no recourse or power over whats happening.
Few things are more important in modern life than access to your money. That's why I would never use an online bank. I use a local credit union but "local" with actual branch offices you can walk into and talk to a human are the important thing.
I tend to think the same way, but I've personally had far more issues with my local (largest in the region) credit union. Their (human) customer support reps frequently make mistakes processing my share certificates that automation would have otherwise avoided.
TD bank has branches but that's not super helpful if you're at the airport. the helpful thing is having a "relationship manager", someone you can call or email directly - you can get that at a big bank or a small one, and you don't need tons of assets, a lot of banks its having your direct deposit with them is enough to qualify for some special relationship.
Pinging the absolute jerkwads at amutable.com who state that it's not their problem how the technology will be used.
I'm in the UK and it always seems weird to me how in the USA its totally normal to publically register yourself as either a Democrat or a Republican. To me seems like the kind of thing that should be private. In the UK I might choose to be a member of a political party but no-one needs to know that except that particular party. Or I can just join no party and vote in private on polling day.
It seems quite weird. Our parties are generally quite weird because were are not supposed to exist in the first place and so most of the rules are sort of ad-hoc.
I don’t belong to one, but it is sort of annoying to not be able to vote in the “primary elections” (where the parties pick their candidates). My state has a strong partisan lean so this is the only election that actually matters here. Oh well.
Because by and large they're all trapped in an "us vs them" team sport where both sides believe everybody on the opposing team is literally Satan.
No, it's because you need to register for a party if you want to vote in its primaries.
I'm really hoping we can get over this silly delusion soon.
Then again, EU citizens would find it questionable how in the UK recruiters are asking about your ethnicity, disability status etc.
This entire thread is such a strong case for why all undergrad CS programs should have a core requirement of at least one ethics course.
You really think taking an ethics COURSE will do anything?
How are you supposed to 'learn' ethics?
Never understood the point of those. My university had over 4 for the cyber students and we all went on to work for the MIC contributing to blowing up brown children in the middle east anyway.
Clearly, our problem must have been that we didn't have 5 mandatory time-waster ethics courses. That surely would have solved the matter. Add in a few extra "racism is bad" and "don't be a meanie" courses to round out the collection, just for good measure.
Every business school and MBA program has ethics course requirements. Does it really make a difference? Maybe it does, and things would be even worse than they are otherwise. But I'm not sure.
Especially if it has concrete historical examples of technology intended for good and how it was later used for evil, rather than something more generally-philosophical.
At least I had 6 humanities courses telling me why communism is good
The automobile, the transistor, the Internet, and basically every other technology ~could~ be used by a police state. I'm not sure how that is a helpful rule at all.
The article says the same. I think the point is to be aware and try to build things in a way to minimize the chance for abuse. There's nuance and the author says as much.
Related:
Don't get distracted (2017) [1] [2]
[1] https://calebhearth.com/dont-get-distracted
[2] https://news.ycombinator.com/item?id=42388354
This is missing the "(2013)" at the end of the HN link, but it does seem weirdly prescient as we say a rise of more polarised politics across the World, just 13 years later.
And a rise in tech companies building products and features that dont benefit the users, knowing we cant really escape at this point.
Thanks, didn't notice it before you pointed it out (and I already made my comment), actually turned into quite an interesting retrospective for me
Clearly you need to consider how general the capabilities and use cases of the technology is. The most general-purpose technologies will always have obvious use cases for police states and other bad groups or people.
The obvious 21st century United States instance of "don't collect data that the government might later use for persecution" is E-Verify.
Defaulting to discussing the US here: let's start with reducing the state's size - it shouldn't be the world's largest employer, it shouldn't be able to run an unfathomably large deficit, it shouldn't be able to declare war without congressional approval. Technology to run a police state is just the last step in a long line of ridiculously unacceptable steps.
Let's slip in your pet bugaboo. Workforce size very little to do with state surveillance. In fact, mandating that it shrink can result in blowback, just like we have seen with how the US provides a great antagonist for China to evolve its own capabilities faster than if we had done nothing.
How does the number of state employees de facto result in a police state?
I agree with you on our obscene debt and our obsession with "conflicts" around the world, but I'd argue we're seeing the mass reduction of state employees and it's just making everything worse (eg massive deregulation and the destruction of checks and balances).
It won’t work for any country. They all want more control, and also create a class of people depending on government money to be on their side. Only people losing out are the powerless ordinary citizens and taxpayers
The sad irony of "Share this post on..." with neat list of all the services doing exactly what the post warns against. They can and often are already used by police state.
Wouldn't a better title be "don't give out information that you wouldn't want used by bad actors"? Seems less about technology and more about privacy.
I've been arguing with folks on Reddit about this lately when it comes to Google's changes to Android (where they've made it more difficult to install any software on your device without Google first tracking the identity of the developer). My argument ultimately comes down to the fact that Google (and Apple already) have built a technology that gives them immense power to control individual computing freedoms, and no matter their internal policy, they still must follow US law. No amount of promises or legit good faith on their part matter, they've made a dangerous technology and are behold to a government that is very eager to censor anything it doesn't like. It's so disappointing to see 1. Google engineers build such a thing, and 2. For people to support it.
Civic Hygiene is the opposite of how you were taught to write code...
>It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter.
https://blog.codinghorror.com/to-serve-man-with-software/
I'd be careful banging two rocks together else someone might start a war with them!
how about you guys understand that state is criminal master org. that uses violence monopoly? is it too high level or hard to understand this?
see war on drugs and more
Has this philosophical approach ever worked? It's a commonplace in the hard sciences and engineering fields (ie, refusing to work on weapons systems), but weapons systems are elite and well-compensated engineering projects that don't have any trouble finding talent. The same thing goes for NSA, which has a line out the door from elite academia.
I'm not dismissing the sentiment here. It's morally sound. You should act in accordance with your principles and it useful to be reminded how those principles intersect the world. I have similar guardrails (probably not the same as the author's). I just wouldn't want people to overestimate what can be accomplished with this approach.
A significant number of people in positions of power in the AI industry get their moral framework from science fiction books and a particular Harry Potter fanfiction. It’s hard to predict what ideas and philosophies will become important.
Sorry, purely just curious here because I'm not sure what you mean - what Harry Potter fanfiction?
"Harry Potter and the Methods of Rationality" written by Eliezer Yudkowsky has become something akin to a sacred text among the rationalist community.
edit: posted is a podcast by Behind the Bastards about the Zizians, a death cult that branched out of rationalist community and which was responsible for six deaths, but it covers the rationalist community and the Harry Potter fanfiction as well.
[0]https://www.youtube.com/watch?v=9mJAerUL-7w
I find it frusrating how the conversation around this almost always focuses on one side of the equation. Why is it always about preventing tools from being built in case authoritarians comes to power rather than preventing those authoritarians from coming to power in the first place? I'm far more scared off the authoritarian aspect than the tech. The Nazis didn't need Flock cameras to kill millions. It seems like focusing on the actual bad guys is going to be more effective because if the state wants to bear its weight on you, it rarely requires tech to do it.
The Nazis relied heavily on IBM technology for national census data (to identify jews), military logistics, ghetto statistics, train traffic management, and concentration camp capacity
https://en.wikipedia.org/wiki/IBM_and_the_Holocaust
I fully agree with your base perspective, that the main focus should be on preventing authoritarians from coming to power in the first place.
I also agree with your final point, that the government doesn't need tech to lean on a person.
We disagree in the middle, on how tech that makes evil orders of magnitude easier might be a contributing factor and might enable the government to lean on many more people than it would otherwise.
See:
* https://dave.autonoma.ca/blog/2019/06/06/web-of-knowledge/
* https://i.ibb.co/WWWYznHX/flock-future.png
* https://www.cfr.org/education/learn/reading/what-fascism
People have incentives to become more powerful - through wealth or in other ways. So unfortunately, authoritarianism tends to have gravity going for it.
The water flows whichever way there's a gradient of height.
Don't create gradients you don't have dams for.
>The Nazis didn't need Flock cameras to kill millions.
No, but they were extremely happy to hop over to France and realize that we kept lists that indirectly listed jews. Actions have consequences that we may not be able to predict.
I remember an old internet post I made in the '90s saying that the internet was like Woody Guthrie's guitar, which had a label on it that read "This machine kills fascists". Where do I go to get my innocence back?
If you build a technology that improves human communication in any way, if successful it will likely be leveraged by a police state.
Libertarianism has its moments but it generally hasn't caught on because state capacity is unique and useful. And virtually every last bit of it can be used to facilitate a police state because it's... a state.
Sure, we could all live without IDs and use motio belli for defence but this arrangement will be soon replaced by someone more organized because there is a huge return to that organization despite its risks.
The article is not completely wrong in its premises, but phrased incorrectly.
This is not about technologies (after all, an authoritarian government will use many existing technologies, from the car to the computer), but about powers granted to a government, that could be used differently by the next party/majority in power.
The answer to that is to not grant government too much power in the first place.
Of course, this is often a contentious point for leftists, since they do want to grant government a lot of power in the first place; they're just concerned that the exact same powers will then be used by a different political side in a way that bothers a different category of people.
You will get downvoted to oblivion for even suggesting it here but it's so obviously a take that maps on cleanly to reality. Massive power centralization in government makes everything dual use in the service of a police state. The easiest thing to do is resist massive centralization of power in a government.
you get downvoted because to most people we don't care if its tyranny.gov or tyranny.com. "don't give powers to the government" just means they buy them from corporations. see: cellphone location data.
the solution is not to have the power exist at all, anywhere.
Or, and here me out…
“Become ungovernable.”
That has been the motto of our governors.
"Civic hygiene isn't about saying we distrust our current government - it's about not trusting the next government." I've made similar points. Granting one government administration more power, is by extension granting it to all future administrations. Do so only with great care.
Like others have said, I don't think it's possible to create technology that cannot be weaponized. I think it's possible (mostly) to design technology so that abuse requires more people, requires the leaving of evidence / paper trail, needs checks, etc.
But still, it's not possible to prevent abuse altogether and sometimes we even applaud breaking the rules (the movie Hunt for Red October for example kicks off with the Soviet captain jumping the guardrails).
Also, the most "benign" public infrastructure is usually the most impactful, e.g., water systems and electrical systems. Don't like the people in that one town who keep protesting and voting against you? Turn off their power, water, and also make it so that their trash doesn't get picked up for a year. See how they like it.
this is an important topic, and i hate that people forget how difficult it is. in any political conversation, it's pretty easy to pull this thread to unravel anyone's adamant belief in one thing or another (e.g. gun control). my biggest concern is that people seem less willing than before to actually have those discussions.. they'll immediately just label you as being opposed to their views, "against" them.. when really we need these conversations to find viable policy solutions that wont easily be used in the exact opposite way that they were intended when implemented
Bertrand Russell:
“If war is not impossible, then every advance in scientific technique in an advance in mass murder”
As true today as then
You can replace war with “police state” and it’s the same message.
https://m.youtube.com/watch?v=GzTlTVaYDtk
Technology is force multiplier. Everything could be used for oppression. Deal with it - the future is boot stomping on human face forever. There are lucky few that could choose whether they will be the face or foot. But it is inevitable.
Or we could stop all tech development. But it is not going to happen.
Edit; wrote this comment before I realised this is a blog post from 2013. In 2013 I would've agreed 110% with this.
-----------
This is one of those things that make for great post title and gets you all riled up at first but when you think about it, it's really shallow.
Most civic databases can be abused by any sufficiently authoritarian government. A homeless shelter's records may contain exactly the sort of information that could be used to get "undesirables" off the street. But those same records are useful for coordinating beds, outreach, medical care, etc. Surely the conclusion isn't that the shelter shouldn't keep records.
It seems much more useful to focus on limiting what gets collected, who gets access to it, and how long it can be retained. In other words, boring things like data-minimisation and retention laws. Somewhat amusingly, that gets you pretty close to the sort of principles behind GDPR, which isn't exactly universally loved around here.
I don't really disagree with the blog post's moral intuition. What bothers me is the emphasis on individual moral choices by the people building these systems. I've seen plenty of my peers happily work for companies that have spent the last twenty years building exactly this kind of technology, there will always be someone willing to build it. That's not an argument for abandoning personal ethics, but it is an argument against treating personal ethics as any form of serious societal safeguard. "good engineers should refuse to build them" means nothing.
Companies that agree to work on Chat Control deserve a special place in hell. One company that was eager to participate, hoping this harmful law would pass, is Atos. Name and shame them.
Assholes like this were all for the police state back when it was up to stop-the-spread “contact tracing”, real-time “fact-checking” my private messages, arresting people for facebook posts, and transing your kids.
Now that the red pedophiles are in charge instead of the blue ones, he changes his tune.
I think this runs into a basic issue that all large software systems that enforce the law contribute to locking down a society because laws tend to need be broken to be changed.
That a system can be avoided offends the investor who want their products to be ubiquitous.
The system that is not perfectly enforced offends the engineer who dislikes broken things.
Yet both are required if we as a society are to grow. I think the thing that it might be good to learn is when to say "I've done/had enough."
> because laws tend to need be broken to be changed
Nope.
I think I could collect enough weight of books on "civil disobedience" at my local library that I could crush you underneath them, even if you didn't bother reading them.
the problem is Capitalism does not need Democracy to function or survive
many countries like Singapore, etc. prove this easily
Authoritarianism can even be desired by Capitalists if they know who to bribe to remove regulations and get approvals
and no surprise, that's exactly where the United States is right now, teetering on the edge of that hole, about to fall in
"AI" is going to be weaponized, already is, and any resistance against its abuse is simply overriden with threats to destroy profits or even company survival at all
> many countries like Singapore, etc. prove this easily
Economic freedom is a necessary but not sufficient condition for political freedom.
well, be glad it's company survival and not personal survival yet. We had that article about the men with guns in egypt recently too
There is some sort of "nerd naivete" that they are the ones with power to control those things.
If you're a nerd developing open source (not an insult; I'm in that category), our job is to develop effective technology.
If you want to do politics, go into law and/or politics (or into business, so you can fund/control politics).
>"But software which OpenBSD uses and redistributes must be free to all (be they people or companies), for any purpose they wish to use it, including modification, use, peeing on, or even integration into baby mulching machines or atomic bombs to be dropped on Australia. " -- Theo De Raadt
What is the point of writing something like this?
It's embarassing for an adult to think like this. There will always be someone willing to build that thing (and that was true even before llms)
> There will always be someone willing to build that thing
… with sufficient time and money. As opposed to the minimal time and money required if the baddies are just pulling an existing solution off the shelf.
I hope you are joking, or haven't thought this through! If you sincerely believe and act on what you're saying here, you should be shunned from polite society.
He's not joking, he's tacitly asking for peace of mind :)
Just because someone is willing to build it doesn't mean it has to be you.
We may all play some part in societies that have some degree of monstrous outcomes, but I feel like staying on the "tragic civilian casualty" side versus the "legitimate target" side of a freedom fighter's bombing campaign is probably the way to go.
It's not about someone willing or not, it's about you, an ostensibly grown adult who should think in terms of your own responsibilities instead of others'.
“Someone else will build it anyway, might as well be me.” I don’t think embarrassment lies at the feet of OP.
It's embarassing for an adult to think like this.
Joe's gonna die anyway. I might as well kill him before someone else does.
It's embarassing for an adult to think like this.
Speaking of embarrassing, your argument is an argument against having any professional ethics whatsoever.
What's embarrassing about it?
Terrible argument. You can say ”why not work as a concentration camp guard? There will always be someone willing to do it so you might as well cash in”
At Nuremberg, they all were just doing their job and executed their orders, those eager beavers of men.
People cite Flock as an example of this. Yet the same people walk around with iPhones and Android devices that can record conversations (who talks to whom at what times and for how long), know our exact locations and log everything we do online.
If I had told you 30 years ago that we are going to build a surveillance network and make the people we are tracking pay for it all you would have laughed me out of the room. But that is what we have built. The modern Smart Phone.
Now we use our iPhones to complain about cameras on public roads as if they are watching us.
Smart phones are opt-in technology. Flock is not.
Very flawed argument, anyone can go for a walk without their phone on them and have a conversation, nobody would ask them why they don't have their phone on them.
I know it is not a popular opinion. But it is my opinion.
Do you seriously think that governments need Flock cameras when they have Apple, Microsoft and Google?
Ostensibly they need a warrant to get that data from Apple/Google whereas they have a direct line to Flock data cause that's how that relationship works
Governments don't need flock or AMG, that's where the flaw in your argument begins.