The narrative around security and LLMs doesn't make sense to me.
From my perspective, this is an emergency because it has been turned into an emergency. I think that everyone involved has the best of intentions, but this is playing out like a greek play. In avoiding what they fear, they've realized that fear.
An example of this is the story of Oedipus Rex, in the story Laius, the king, is told that he is "doomed to perish by the hand of his own son." (and wed his mother) And so to avoid this fate he decides to kill the infant. The person assigned to abandon him in the woods takes pity on the baby and gives the baby away. Thereby ensuring that Oedipus knows neither his mother or his father, and giving him one of many reasons to kill him.
The child grows up and hears the same prophecy again and the child tries to avoid the prophecy as well, as he loves his adoptive parents. So he leaves them and travels to Laius' kingdom, where he runs into Laius. Neither recognizes the other. They get into an argument and Oedipus ends up killing him.
I think the ancients were on to something, because if Laius had reacted to the prophecy with courage, he would have been saved. I would like to argue that if he had faced his fear and raised Oedipus with love, then the necessary preconditions for the prophecy to come true wouldn't have taken root. But that's not what happens.
They're driven by their neuroses and so in acting with cruelty out of fear, Laius makes the prophecy real.
I think a lot of people in this AI research sub-culture would be served well by reading Sophocles' play, because they are making their self-prophesied doom come true.
They have been convinced for years (GPT-2 was released in Feb 2019) that AI is dangerous. A tremendous threat. An apocalyptic threat.
One dimension of this fear has been the idea that a super smart AI will take over our digital infrastructure and be responsible for the digital apocalypse. And so they taught these models how to exploit vulnerabilities.
They are so convinced that it's dangerous that they start testing it as if it was a weapon with offensive capability.
And as they don't want to release a weapon (oh no!), they restrict access to their AI, thereby depriving everyone of a valuable tool they can use to improve their security.
>The Work Number is an Equifax (who famously had a massive data breach a few years back) owned database with employment and pay information.
>You can create a login on theworknumber .com and pull your report. Mine is seventy four pages and had info from every company I've worked at in the last 13 years including every paycheck I had received with the exact dollar amount (both net and gross) and hours worked. It has employment start/end dates, termination reason, details about withholdings, benefit enrollment, union affiliation, etc etc etc.
>This data is sourced directly from the HR platform your employees use (ADP, Rippling, Gusto, iSolved, etc). Equifax sells your data for things like employment background checks.
>You cannot have your data removed from The Work Number. You can freeze your report (much like a credit report) but if a potential employer cannot access your frozen report that may disqualify you.
>Why YSK: If you're interviewing for a job you can be at a significant disadvantage especially for things like salary negotiations because they can literally see how much you make including your most recent paycheck. Creditors also can access these reports.
>This is the biggest privacy violation I have ever seen and almost nobody is even aware of it. Certainly none of us consented to having our employment and income data harvested and sold, especially since we get nothing in return. At a minimum, people should know about how this data can impact you.
>Edit: The Work Number is primarily for the US, but there are similar services for other countries.
> I honestly don’t blame them: [...] software is software
That attitude is the problem. Why does our industry have that attitude towards quality? Every bike shop in my little town is better with quality than the average software shop in the world. Yes, software is more complex than bicycle. But a software engineer also gets paid 10x and has the luxury of spending substantial time on their product, compared to the 10 minutes it takes the bike guy down the street to diagnose and then fix an issue with my bike which I then trust my life with once they are done and I bike through traffic.
We need to treat software differently. "Oh well, it's just software shrug" does not cut it anymore, if it even ever did.
Probably because software can be adjusted at any time on the fly. Especially web applications. Little incentive to get it right from the start and big incentives to keep meddling with what's already working. The bike shop's quality might also deteriorate if it could instantly rollout patches to all customers after the purchase.
The incentives change if there's legal repercussions to leaking data. Data breaches can lead to impacts on personal safety and finances, but I think some crisis or serious incident needs to happen before the average person realises this and makes this a big enough political issue.
Is it just me that has a constant dribble of other people's private data coming to me from government agencies?
This year, I've had the UK variously send me someone else's court summons, someone else's tax documentation (not address error - just inadvertently included as attatchments on emails to me).
I've had portugal disclose property taxation information on any subject to me, through a simple enumeration hole.
And last week I had brazil's ministry of agriculture send me details on other peoples' livestock shipments.
Our information environment is unbelievably porous. There would be no secrets from an unshackled AI.
- a UK police force send me a speeding prosecution notice for a car I'd sold 6 months prior to the offence.
- a UK energy company send me to a debt collection agency for unpaid bills they had continued to send to a non-existent address even after telling me they had cleared what was owed and corrected their records of the address.
- a German government official showing up at my door to ask if I was running a specific named business (that I'd never heard of) at the house, which I'd only just taken ownership of from the builder
Someone I used to know in the UK kept getting emails for other people with the same name, including lawyer confidential communications for a namesake who was involved in one of the big banking scandals.
> Our information environment is unbelievably porous. There would be no secrets from an unshackled AI.
Absolutely. My biggest defences against that include the luck of someone else more famous with my own name.
i am also worried that all my private data might eventually (once tech is advanced enough) be surfaced somewhere: photos, chats, emails. wondering if i should start cleaning up now or just give up
Is "ML" some UK-ism? I sort of see the connection between machine learning and LLMs but it doesn't seem too obvious to me. And certainly it's not the common nomenclature.
Way to get hung up on a tangent. Yes LLMs are ML models, they are trained with an unsupervised learning objective, then fine tuned with supervised learning and then reinforcement learning, which are all the three main branches of machine learning. It is trained on a training set, with optimizing a training loss, with a learning algorithm. It's as ML as it gets. Do you only want to call cat vs dog image classifiers ML? Or only SVMs?
AI is a superset of ML, though today most successful AI approaches are based on ML so the line has blurred in casual speech.
ML (machine learning) became a less fashionable term, so AI took over again (having previously fallen out of favour). The terms are essentially used interchangeably, with the idea of real AI now being referred to as AGI (Artificial General Intelligence) (not GAI or GenAI, as those now stand for Generative AI, which includes LLMs, diffusion models, etc.)
All ml is AI, based on the use of the term AI for many decades. For many problems I think it’s more descriptive (learning the rules from data rather than being shown them) and not all classical AI is ML (path finding for example). LLMs are absolutely ML and AI as far as tradition goes and personally I think are one of the very few things that are AI as regular people might have thought it meant back when it was much more obscure (I was into it before it was cool dontchaknow, an AI hipster).
The narrative around security and LLMs doesn't make sense to me.
From my perspective, this is an emergency because it has been turned into an emergency. I think that everyone involved has the best of intentions, but this is playing out like a greek play. In avoiding what they fear, they've realized that fear.
An example of this is the story of Oedipus Rex, in the story Laius, the king, is told that he is "doomed to perish by the hand of his own son." (and wed his mother) And so to avoid this fate he decides to kill the infant. The person assigned to abandon him in the woods takes pity on the baby and gives the baby away. Thereby ensuring that Oedipus knows neither his mother or his father, and giving him one of many reasons to kill him.
The child grows up and hears the same prophecy again and the child tries to avoid the prophecy as well, as he loves his adoptive parents. So he leaves them and travels to Laius' kingdom, where he runs into Laius. Neither recognizes the other. They get into an argument and Oedipus ends up killing him.
I think the ancients were on to something, because if Laius had reacted to the prophecy with courage, he would have been saved. I would like to argue that if he had faced his fear and raised Oedipus with love, then the necessary preconditions for the prophecy to come true wouldn't have taken root. But that's not what happens.
They're driven by their neuroses and so in acting with cruelty out of fear, Laius makes the prophecy real.
I think a lot of people in this AI research sub-culture would be served well by reading Sophocles' play, because they are making their self-prophesied doom come true.
They have been convinced for years (GPT-2 was released in Feb 2019) that AI is dangerous. A tremendous threat. An apocalyptic threat.
One dimension of this fear has been the idea that a super smart AI will take over our digital infrastructure and be responsible for the digital apocalypse. And so they taught these models how to exploit vulnerabilities.
They are so convinced that it's dangerous that they start testing it as if it was a weapon with offensive capability.
And as they don't want to release a weapon (oh no!), they restrict access to their AI, thereby depriving everyone of a valuable tool they can use to improve their security.
Relevant:
https://www.reddit.com/r/YouShouldKnow/comments/1wssf4u/ysk_...
https://imgur.com/a/6FaQQhb (original post before being taken down by mods)
>The Work Number is an Equifax (who famously had a massive data breach a few years back) owned database with employment and pay information.
>You can create a login on theworknumber .com and pull your report. Mine is seventy four pages and had info from every company I've worked at in the last 13 years including every paycheck I had received with the exact dollar amount (both net and gross) and hours worked. It has employment start/end dates, termination reason, details about withholdings, benefit enrollment, union affiliation, etc etc etc.
>This data is sourced directly from the HR platform your employees use (ADP, Rippling, Gusto, iSolved, etc). Equifax sells your data for things like employment background checks.
>You cannot have your data removed from The Work Number. You can freeze your report (much like a credit report) but if a potential employer cannot access your frozen report that may disqualify you.
>Why YSK: If you're interviewing for a job you can be at a significant disadvantage especially for things like salary negotiations because they can literally see how much you make including your most recent paycheck. Creditors also can access these reports.
>This is the biggest privacy violation I have ever seen and almost nobody is even aware of it. Certainly none of us consented to having our employment and income data harvested and sold, especially since we get nothing in return. At a minimum, people should know about how this data can impact you.
>Edit: The Work Number is primarily for the US, but there are similar services for other countries.
I was hoping this would give me an exhaustive log of my work history but it has roughly 3% of my career, I froze it anyways
Yeah I imagine how much it has will vary wildly from person to person, but some it has pretty much everything down to anal circumference.
> I honestly don’t blame them: [...] software is software
That attitude is the problem. Why does our industry have that attitude towards quality? Every bike shop in my little town is better with quality than the average software shop in the world. Yes, software is more complex than bicycle. But a software engineer also gets paid 10x and has the luxury of spending substantial time on their product, compared to the 10 minutes it takes the bike guy down the street to diagnose and then fix an issue with my bike which I then trust my life with once they are done and I bike through traffic.
We need to treat software differently. "Oh well, it's just software shrug" does not cut it anymore, if it even ever did.
Probably because software can be adjusted at any time on the fly. Especially web applications. Little incentive to get it right from the start and big incentives to keep meddling with what's already working. The bike shop's quality might also deteriorate if it could instantly rollout patches to all customers after the purchase.
The incentives change if there's legal repercussions to leaking data. Data breaches can lead to impacts on personal safety and finances, but I think some crisis or serious incident needs to happen before the average person realises this and makes this a big enough political issue.
Is it just me that has a constant dribble of other people's private data coming to me from government agencies?
This year, I've had the UK variously send me someone else's court summons, someone else's tax documentation (not address error - just inadvertently included as attatchments on emails to me).
I've had portugal disclose property taxation information on any subject to me, through a simple enumeration hole.
And last week I had brazil's ministry of agriculture send me details on other peoples' livestock shipments.
Our information environment is unbelievably porous. There would be no secrets from an unshackled AI.
Not specifically as you say, but I have had:
- a UK police force send me a speeding prosecution notice for a car I'd sold 6 months prior to the offence.
- a UK energy company send me to a debt collection agency for unpaid bills they had continued to send to a non-existent address even after telling me they had cleared what was owed and corrected their records of the address.
- a German government official showing up at my door to ask if I was running a specific named business (that I'd never heard of) at the house, which I'd only just taken ownership of from the builder
Someone I used to know in the UK kept getting emails for other people with the same name, including lawyer confidential communications for a namesake who was involved in one of the big banking scandals.
> Our information environment is unbelievably porous. There would be no secrets from an unshackled AI.
Absolutely. My biggest defences against that include the luck of someone else more famous with my own name.
As opposed to Big Tech sharing your data ?
While that was never good, it’s still better than your data being completely leaked to the public.
i am also worried that all my private data might eventually (once tech is advanced enough) be surfaced somewhere: photos, chats, emails. wondering if i should start cleaning up now or just give up
Is "ML" some UK-ism? I sort of see the connection between machine learning and LLMs but it doesn't seem too obvious to me. And certainly it's not the common nomenclature.
Way to get hung up on a tangent. Yes LLMs are ML models, they are trained with an unsupervised learning objective, then fine tuned with supervised learning and then reinforcement learning, which are all the three main branches of machine learning. It is trained on a training set, with optimizing a training loss, with a learning algorithm. It's as ML as it gets. Do you only want to call cat vs dog image classifiers ML? Or only SVMs?
AI is a superset of ML, though today most successful AI approaches are based on ML so the line has blurred in casual speech.
> I sort of see the connection between machine learning and LLMs but it doesn't seem too obvious to me.
Why isn't it obvious? Transformers are deep learning models, and deep learning is a subset of ML.
ML (machine learning) became a less fashionable term, so AI took over again (having previously fallen out of favour). The terms are essentially used interchangeably, with the idea of real AI now being referred to as AGI (Artificial General Intelligence) (not GAI or GenAI, as those now stand for Generative AI, which includes LLMs, diffusion models, etc.)
Machine Learning has always been in the vernacular.
I think Apple used to use this a lot in their presentations until 1-2 years ago to mean "AI".
All ml is AI, based on the use of the term AI for many decades. For many problems I think it’s more descriptive (learning the rules from data rather than being shown them) and not all classical AI is ML (path finding for example). LLMs are absolutely ML and AI as far as tradition goes and personally I think are one of the very few things that are AI as regular people might have thought it meant back when it was much more obscure (I was into it before it was cool dontchaknow, an AI hipster).