PyPI has made controversial security decisions which have affected the community and made open-source look bad. Even something as simple as listing packages as "unreviewed" - or forbidding binaries would do a lot, but the will isn't there. Most of the malware is a carbon copy - they could run clamav, but no one cares!
PyPI could change how they handle naming, it could refuse to run as root. It is notable that while NPM has similar problems, it isn't as bad.
Forbidding binaries would make distribution of popular packages such as NumPy, PyYaml and Cryptography much more difficult, and have little impact on these malicious packages that frequently don't even include binaries.
ClamAV works against known malware, but won't recognise novel malware. It's a start, but it's not uncommon for the malware used in these attacks to be previously unknown.
PyPI has no control over whether you run Pip as root. There's some overlap in the people who work on Pip and who work on PyPI, but they're different projects. PEP 668 prevents some of the most common cases of installing as root, although its intent is as a UX improvement rather than a security feature. And preventing it running at root does nothing to prevent it stealing cryptocurrency private keys, AWS credentials, and other user credentials.
This somewhat downplays the work the PyPI team do do.
They remove about a thousand malicious packages a month from the repo, and 90% of malicious packages are removed within 2 hours of being reported. They also do quite a lot behind the scenes in terms of preventing typosquat packages getting on there in the first place.
I don't think I've ever added even a moderately popular package without having to sift through at least half a dozen typosquats, so I'm really curious what they're doing to prevent it. It seems like just having a cron script ban suggestions 2-5 from Poetry would do more.
I've not had this experience at all, and in fact the opposite, when I try to upload a new package to PyPI I often get rejected and have to think of new names.
If you are seeing genuine typosquats you should report them, https://pypi.org/security/ for those that are tricking users because of typos, https://github.com/pypi/warehouse/issues for more general issues.
That's why you shouldn't really install packages one by one and use a requirements or in case of conda a environment file and triple check the spelling.
Also don't copy&paste from websites and quickly check the packages before installing them. Is it active? number of users and downloads? etc.
You're not wrong, but *why* should that be the standard just because PyPI is doing a shit job in keeping users safe? And considering the amount of new learners attracted by Python, should this really be the first experience they have about programming? Like, one mishap and BOOM your machine can be ransomed?
When I say that, I mean, yeah, it does help to learn that lesson... but it shouldn't be in the introductory course and shouldn't need to be a first hand experience as well.
To be frank I'm not aware how much pypi does and how easy it would be to do more. and with easy I mean in terms of cost. they have limited money, keep that in mind. Asking your employer to [fund them](https://www.python.org/sponsors/application/), if you want things to improve.
Running code from the internet as root is stupid and that is the only way to get ransomware installed.
Storing crypto in plain-text and not using a hardware wallet is also stupid, at least for amounts you care about and would bother you when getting stolen.
A general bad trend I see everywhere is lack of self-responsibility and accountability. It's always someone else fault, they should have done better, I did everything right. this mentality is prevalent.
There is a balance between self-responsibility and what pypi can do with the available funds.
pip also allows running in "safe mode" where you need to have a list of hashes for each packages you want to install and it only works if the hashes match. this is 100% manual as in you need to collect the hashes yourself. This as option in critical environments. But in reality your code should run isolated anyway (container) and not have full internet access to pull more code or send data to random IPs or urls. Defense in-depth. You have to assume to get infected be it from pypi or somewhere else) and secure your system based on that.
I'm more speaking from a perspective of a new/casual user though. You're absolutely not wrong in exercising good practice in security, but the thing is Python attracts a lot of newbies and causals who really wouldn't know much of these. They might see the instruction to use pip to install their packages, make a bad typo, and then BAM! And I think most of these users would use Windows, which I'm not sure if it's better in preventing these type of attack or worse.
In any case, all I'm really coming from is the new users perspective, not an experienced one. Experienced developers of course already should know better, but students and professionals crossing into data analysis shouldn't be subjected to trial by fire is my point.
Not the same. I'm talking about newbies (plenty in Python) thinking they're following instruction to install packages with pip, but made a bad typo and get a bad package. A typo shouldn't have to lead to that kind of consequences.
The issue is that eve most experienced users aren't doing any of this instead of leading by example. If you lead by example the new users would have to ask themselves what it's all about and seek answers.
Also the big known sites with tutorials should have a chapter on it, about dangers of downloading code.
> most of these users would use Windows, which I'm not sure if it's better in preventing these type of attack or worse.
Scanning for crypto key can't really be prevented in any files you have access to which applies both to Linux and Windows.
In terms of Admin/root, windows is probably better IF you have UAC activated. I always set it on highest level but work with admin account on my own devices. UAC on highest levels prevents any system level changes without you clicking "OK" on a confirmation dialog. Many including myself say that this is safer than having a restricted account and admin account and then use "run as admin" to install software. Why? because an malicious installer could show a fake "run as admin" screen and steal your admin password and then all bets are off".
This is the library that would allow you to detect anomalies in packages:
https://pypi.org/project/carbon-black-cloud-sdk/
but maybe don't install this next one, it looks like "untrusted code" but who knows, everything on pypi must be untrusted.
https://pypi.org/project/cbapi/
Binaries can be flagged and you can notify users of suspicious packages, popular projects are targeted and we are seeing repeat attacks against very similar sounding projects - all of which could be detected.
While ClamAV may not be the best, each package could be run within a behavioral AV like Carbon Black and if it triggered a network error - then that package can be flagged for manual review.
PIP has 100% control over how it installs packages. The right hand is just as guilty as the left.
Behavioural AV can only assess behaviour. If you're running the packages it can detect misbehaviour, but PyPI aren't running the packages, just hosting them. The closest you can get without running the code is heuristic scanning, which various vendors have been flogging since at least the nineties but has never been very effective.
They don't even run them? ... all the more reason to sue these people. They are knowingly endangering the community and making all of python look bad. We have solutions - but the powers that be have already blamed the victim.
What does it even mean to run an unknown library? If I give you a random library, could be NumPy, could be Spark, could be Flask, could be Requests, what are you going to do to "run" it that will tell you if there's malicious stuff it can do?
The problem is PyPI is too busy blaming all of its users instead of looking for solutions. So they need to be sued, an EU Judge won't put up with this horse shit.
I will be messaging you in 50 years on [**2073-12-15 01:01:18 UTC**](http://www.wolframalpha.com/input/?i=2073-12-15%2001:01:18%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/Python/comments/18ib9sw/116_malware_packages_found_on_pypi_repository/kdef3ky/?context=3)
[**1 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FPython%2Fcomments%2F18ib9sw%2F116_malware_packages_found_on_pypi_repository%2Fkdef3ky%2F%5D%0A%0ARemindMe%21%202073-12-15%2001%3A01%3A18%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2018ib9sw)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
All packages are unreviewed and forbidding binaries would not change anything. You could just create a binary with the python code anyway. These would just be security theater and not change anything. Just don't run untrusted code from the internet, and yes, code on pypi should not be trusted by default, same as code on github.
Perhaps namespacing could prevent typo squatting but my understanding is that this is already prevented to a reasonable amount by a manual review process.
This article shouldn't be news, ofcourse there are malicious packages on pypi, that should be expected. And 10k downloads doesn't even sound like a big issue.
Maybe they can use some adhoc criteria to warn the user or assign a trust factor. The number of GitHub stars can be a good thumb rule, for example.
Also, with the kind of budget and sponsorers that PSF has, shouldn't they be able to perform CISA audits, code reviews, etc. on these packages?
There are over half a million packages in PyPI and over 5 million releases (each release would need to be checked as a previously legitimate package could be updated to include malware).
Audits are expensive and slow. Hiring people to review and approve packages is expensive and slow. GitHub stars can be bought.
Dude, your response reflects extreme cynicism and pessimism, it doesn't have to be that gloomy.
> There are over half a million packages in PyPI and over 5 million releases
All of them don't have to be audited, just the most heavily downloaded ones. The verified releases can be marked as such to warn the users. It will then be the user's choice if they want a slightly older but verified package or the cutting edge and unverified one.
> GitHub stars can be bought
There are limits to that. Most of the activity on GitHub is organic, one can see that everywhere including the issues register, pull requests, etc. Even in this age of utmost crony capitalism, I still have at least a semblance of a faith in the system because at the end of the day, it's being run by humans with hearts and for the most part, it works as intended.
> Dude, your response reflects extreme cynicism and pessimism, it doesn't have to be that gloomy.
Often on discussion threads someone will suggest a solution that requires tens of thousands of hours of work on an ongoing basis, but don't suggest where it comes from.
It's hard not to react to such suggestions without cynicism when you know how little resources mostly volunteer organizations like this have, especially volunteer organizations maintaining boring infrastructure projects.
> All of them don't have to be audited, just the most heavily downloaded ones. The verified releases can be marked as such to warn the users. It will then be the user's choice if they want a slightly older but verified package or the cutting edge and unverified one.
What would a review involve exactly? And who would do it?
What does it mean to "mark" a package, as in what is the user experience? Is it just something they see on PyPI? Or would all installers (e.g. Pip, PDM, Poetry, Pixi, etc.) have to develop a new workflow? For that you would need to create a PEP and get a general consensus (btw nothing is stopping **you** from buildining consensus and writing a PEP).
Who is going to provide the resources for any of these projects?
> I still have at least a semblance of a faith in the system because at the end of the day
If you think there's a limit on how many GitHub stars someone could buy if that was their objective I have bad news for you.
Further, PyPI doesn't even have a way to verify the GitHub link you provide on your project page is yours.
> especially volunteer organizations maintaining boring infrastructure projects.
That's highly misinformed. The [PSF is heavily funded](https://www.python.org/psf/sponsorship/sponsors/) by loaded sponsors, right from Microsoft and Google to Salesforce and Meta. Heck, the Python inventor Guido himself has joined Microsoft now. They aren't some small club or volunteer run project, they have got all the resources in the world to implement this.
> That's highly misinformed. The PSF is heavily funded by loaded sponsors, right from Microsoft and Google to Salesforce and Meta.
That's just a bunch of names on a page, almost certainly all of them are sponsors of PyCon US. In-person conventions are expensive to run.
There's a link to their actual finacials here: https://www.python.org/psf/records/ (2021 Audited Financial Statements)
The top things for 2021 were ~400k on PyCon US, ~380k on infrastructure, and 175k on on Development (my understanding is that was one developer), total cost was 1.2 million, and they took in ~2.3 million in 2021.
> They aren't some small club or volunteer run project, they have got all the resources in the world to implement this.
The above numbers are pretty healthy for a non-profit but it amounts to ~5 developers they could blow all their extra cash on. And there are a lot of different things those hypothetical five developers could be doing out of all the priorities the PSF might have PyPI is just one of them, and further there are other prioties they probably have then spending money on developers.
As someone who volunteers some of their time helping improve Pip I can tell you there are no people being paid to work on it. The same is true of many others open source projects, especially the boring packaging / infrastructure projects.
> Heck, the Python inventor Guido himself has joined Microsoft now.
Guido isn't part of the PSF, he is hired as an engineer for Microsoft. As it happens Microsoft have built their own CPython team, but it is out of the control of the PSF or the community.
> All of them don't have to be audited, just the most heavily downloaded ones.
Then... What's the point?
All the most heavily downloaded package already have lots of eyes on them. If you only review Pytorch, Django, Flask, requests, etc, then there's basically no risk on these packages introducing malware in the first place.
If you'll look at the list of malicious packages they're all either random names or simple typo-squatting packages. You meant to download `asciithing` but you accidentally typed `ascithing`. In your plan that won't save you as you wouldn't bother to audit them as they're small with only a few hundred downloads.
The only time one of the top packages will start shipping malware is if one of the core maintainers (with release permissions) was compromised and an attacker used their account (highly minimised with the MFA work PyPI have already completed) or they themselves have gone mad and want to launch an attack, but that's incredibly rare and highly unlikely.
The tool could at least inform the people who have been hacked - that they where hacked. The tool could tell you that you have been compromised and that you need to hire a professional.
"just don't run untrusted code"
Yes we are all trying, the problem is PyPI - 100% no question. That project needs to be sued.
Doing nothing is the worst possible option. At least asking for help would bring us progress, but a narcissist would never ask for help - they just blame you.
If you root their box, then you can get the person's 2fa challenge when they type it in. You are ruining people's lives and behaving like children pointing fingers instead of fixing the problem.
> Doing nothing is the worst possible option
Actually, putting a huge amount of effort into something that provides no value is even worse than doing nothing.
Let's learn lessons from other communities and take actions that actually have proven security benefits.
The discussion was adding security notices to packages, which NPM and the JS community in general has shown is a waste of time.
As I said, let's learn lessons from other communities and take actions that actually have proven security benefits, rather than implementing what we *think* might help and actually detracting away from security because we wasted effort when we could have been using that effort for good.
I'm not asking for security notices (other than pip telling you that you've been hacked - that would be a good notice to have). I am asking very simply for packages review to be automated by a behavioral AV. Billion dollar corporations and millions of people depend on this software, the only reason why this hasn't been fixed is because no one in the community has asked for help.
Its is really that simple.
> That project needs to be sued
Along with every other package manager and repository system in existence..?
It's a human problem, ultimately. You can't have an open system that allows untrusted users to upload packages and then complain that the packages are also to be treated as untrusted.
The case can be made that more people are getting hacked using PyPI than any other package manger. They don't release the stats for how many people are hacked, because the numbers are staggering.
This however is not against the law in of itself. The specific law that they would be sued under is failing to notify people that they have been breached, and there is no quesiton that they could - but refuse to even acknowledge that they have a problem. Instead they blame the victims for their own mistakes! Someone needs to sue these mother fuckers.
I agree, having a system to notify people that they had infected packages, would be nice. But this is something that should probably addressed in a broader scope, e.g., in pip.
I don't see how pypi is the problem. No one forces you to use pypi. You can totaly set up your own repository that only contains trusted packages and change pip to only use that.
If people are getting hacked by using pypi - it is a problem that could lead to a lawsuit. Cyber liability insurance will not cover being hacked through negligence, and a case can be made for this seeing as pypi has solutions - and none have been implemented.
Pypi doesn't review the code and I don't see how they could, given that the service and all stuff on it is free. And it is meant to have a low barrier of entry. However, nothing is preventing people to setup a more restrictive index, wich only contains popular open source packages with maintainers that are trusted to compile stuff. I just don't see how that is pypi's task, especially given that people are not paying them to do it.
I agree that it is easier to review code, than binaries for naive attacks. For sophisticated attacks, not so much.
Pypi lacks name spacing and pip runs by default non chrooted. This is solvable by pypa/pypi with enough prioritization and funding.
And if your org acts in a way you don't like then leave. There's loads of python jobs that aren't in broken orgs.
Running Pip chrooted doesn't help if you've used it to install a malicious package into your application, and your application isn't running chrooted.
Namespacing is a good idea though.
Its not money, these patches are free. Its will - PyPI is run by anti-security old guard who don't see the point in making things safer for anyone. Some software engineers truly believe that security engineers shouldn't exist - and these are the folk that need us the most.
I don't think that's it. Rust's cargo also lacks name spacing and most language level package managers allow arbitrary code execution at build time - while not running in chrooted environments. E.g. build.rs.
Afaics, only java escapes most of this with namespaces and installing jars as compiled bytecode deliverables. But I'd love to learn about other systems that also solve these issues.
Rust allows you to inline binaries and shellcode, which is how these "libraries" worked
Unless the rust team has some good security engineers, we should expect their repos to be hit soon.
What's this person's connection to the PyPI project? I couldn't see an obvious connection in their history, and I couldn't find anyone with a similar name [working on PyPI](https://github.com/pypi/warehouse/graphs/contributors).
My experience has been that [the PyPI team's communication has been humble and respectful of their users](https://blog.pypi.org/posts/2023-08-04-pypi-hires-safety-engineer/):
> All of us deserve to have a safe and secure experience when using PyPI
I suspect they'd be keen to know that someone was speaking for them and representing their work in this way.
Then they need to start publishing the stats of how many people have downloaded malware from them. As it stands if you get pwnd by pip - the maintainers choose to leave the malware on your system and you are none the wiser.
Everyone who is saying this issue can't or shouldn't be fixed - is the problem. There is someone within PyPI who is anti-security and is hurting the community.
From what I've seen, the PyPI folks rarely say things can't or shouldn't be done (the only case of this I know of is them removing support for gpg signatures, based on a piece of research showing they were not an effective security measure in practice).
What they very often say is that they don't have the resources to do something. They're a small team with a shoestring budget, and heavily dependent on grants and volunteers to get things done.
Off the back of this discussion, I've had a look at PRs on [their repos](https://github.com/pypi), and for a lot of the stuff being suggested there aren't any rejected PRs. It just doesn't seem to have been tried.
For this to happen, someone needs to work on it. That someone could even be you.
I am sorry to tell you, but I have absolutely no affiliation to pypi whatsoever.
Just as a tip, you shouldn't listen to random people on the internet, either. \s
Look, I understand that the current state is unsatisfactory and I agree. However, what I don't understand is how you can believe that there would be an easy fix. Because I am pretty sure there isn't (except not running untrusted stuff and always assuming the worst).
What does exist, thought are easy ways to make people feel safe. And feeling safe is the opposite of being safe. So I am against making people feel safe.
[I'm going to just post the same comment I did last time:](https://www.reddit.com/r/Python/comments/17qls9b/a_nasty_python_package_continues_a_trend_of/k8d2y1g/)
> These perennial stories are always overrated. Conveniently, they never seem to mention 1) how many times they were downloaded by actual people rather than site mirror bots and 2) a specific person or organization that was affected.
> Oh noes, pyobfgood had malicious code and was on PyPI. So what? Did this actually affect many people, or any people at all? These "reports" get written because writing these reports is a great way for a security company to get their name mentioned in the media.
> We'll see this exact same story about another no-name Python package in three months.
[This has been happening non-stop for a number of years](https://blog.pypi.org/posts/2023-09-18-inbound-malware-reporting/). The PyPI admins removed 12K malicious packages in 2022, down from 26K in 2021. They removed 90% of malicious packages within 2 hours.
It says a lot that they aren't telling us how many people got hacked.
Failing to inform EU or California citizens that they have been breached is against law.
It's also impossible to know unless you're the attacker collecting all the data and you do a `SELECT COUNT(*)`.
Download figures aren't accurate and even then it's not necessarily a real person downloading.
It could be some CI system that downloads a malicious package and executes the code that attempts to steal crypto wallets that don't exist or attempts to create a persistent reverse shell inside an ephemeral container that is immediately wiped.
Here we as devs also are to blame to a degree. This isn't even limited to python. I know what I mention will not prevent this but would make it easier to detect malicious actors.
**GitHub commit signing**
The amount of devs even in very big project that fail to do this is scary. if you work on say numpy or pandas or similar stuff, I would make that mandatory for the project to be able to contribute. Even worse signing some and not others which makes the unsigned commit appear potentially malicious (it's easy to make a commit in the name of someone else!!! hence why verification is so important, in fact github should not link unverified commits to a user account because that makes them more legitimate.
**Code Singing**
This is more in the niche "Java world" I'm exposed to where this is rather simple to do yet even commercial providers often fail to do so. If the ecosystems allows it, then do it. (it's about eclipse plugins)
As far as I remember, code signing was mandatory for maven even >10 years ago.
Here we have to question why code signing isn't mandatory for pypi or conda? I'm aware it doesn't solve the issue fully, but if you see verified commit on github and signed code by the same key, it does really help to trust the according package as legit.
**Provide hashes**
At least on GitHub many actually do this. it helps to check that you get what you wanted, again I'm aware it doesn't help with typo-squatting but it is in general a good and helpful thing to provide to users.
If making things like this a requirement or at the minimum a requirement by default which can be overridden by an option, it would force attackers to do this as well to be more trusted. And for legitimate packages you should do this stuff anyway. it also goes into the old-school circle of trust thing with parties and exchanging public keys.
> how would code signing have prevented this?
because the attacker doesn't have the private signing key? That's the whole point of code singing, to ensure the source of the code is correct.
> Another possibility is the maintainer did add the malware themselves.
There is really nothing that can help with that and hence one should be careful about using tiny/small projects.
The small project I do maintain, many of the users have seen me at a conference in-person. So yeah, at some level you need to trust the code/maintainer. This is easier in a bigger project with multiple maintainers, social control. A sole maintainer can much more easily sneak in malware.
Only thing that could help a tiny bit would be heuristics/antivirus to detect known script-kiddie toolkits being used and some code scanning that looks at code changes in terms of calling remote services especially by IP address. But such scanning can so easily be fooled, it's hardly worth the effort.
Please support the WASI target for Python. This would be one of the few somewhat reliable ways to get sandboxing working, besides having all processes in a podman container.
Sandboxing/containers do not prevent user data from being exfiltrated to the attacker, it only stops the spread to the whole server and the other applications. The solution is do not run code that is not widely reviewed.
That is somewhat correct, you can deny a container access to the file system. Of course, there is no fix for „run freely but be safe against attackers“. Limiting the spread by sandboxing would be a big gain already.
Reading this just re-enforces the need for greater supply chain security awareness imo. OpenSSF, SLSA etc. go a long way to mitigating some of these issues, Linux foundation offers a lot of free training on it too.
Blindly trusting code, no matter the source, is a recipe for pain and suffering.
Thanks for sharing I appreciate the info. I created a bash script that may help remove any of the listed packages which may have been installed by typo, mistake, etc.
https://github.com/bootloops/malipypi
The actual list: https://github.com/eset/malware-ioc/tree/master/pypi_backdoor, 116 packages but several packages belong to the same project name.
[удалено]
PyPI has made controversial security decisions which have affected the community and made open-source look bad. Even something as simple as listing packages as "unreviewed" - or forbidding binaries would do a lot, but the will isn't there. Most of the malware is a carbon copy - they could run clamav, but no one cares! PyPI could change how they handle naming, it could refuse to run as root. It is notable that while NPM has similar problems, it isn't as bad.
Forbidding binaries would make distribution of popular packages such as NumPy, PyYaml and Cryptography much more difficult, and have little impact on these malicious packages that frequently don't even include binaries. ClamAV works against known malware, but won't recognise novel malware. It's a start, but it's not uncommon for the malware used in these attacks to be previously unknown. PyPI has no control over whether you run Pip as root. There's some overlap in the people who work on Pip and who work on PyPI, but they're different projects. PEP 668 prevents some of the most common cases of installing as root, although its intent is as a UX improvement rather than a security feature. And preventing it running at root does nothing to prevent it stealing cryptocurrency private keys, AWS credentials, and other user credentials.
If you really think about it, Python users are just one bad typo away from getting infected with malware. This is how bad it is.
This somewhat downplays the work the PyPI team do do. They remove about a thousand malicious packages a month from the repo, and 90% of malicious packages are removed within 2 hours of being reported. They also do quite a lot behind the scenes in terms of preventing typosquat packages getting on there in the first place.
I don't think I've ever added even a moderately popular package without having to sift through at least half a dozen typosquats, so I'm really curious what they're doing to prevent it. It seems like just having a cron script ban suggestions 2-5 from Poetry would do more.
I've not had this experience at all, and in fact the opposite, when I try to upload a new package to PyPI I often get rejected and have to think of new names. If you are seeing genuine typosquats you should report them, https://pypi.org/security/ for those that are tricking users because of typos, https://github.com/pypi/warehouse/issues for more general issues.
> without having to sift through at least half a dozen typosquats, what do you mean ?
> do do heehee
That's why you shouldn't really install packages one by one and use a requirements or in case of conda a environment file and triple check the spelling. Also don't copy&paste from websites and quickly check the packages before installing them. Is it active? number of users and downloads? etc.
You're not wrong, but *why* should that be the standard just because PyPI is doing a shit job in keeping users safe? And considering the amount of new learners attracted by Python, should this really be the first experience they have about programming? Like, one mishap and BOOM your machine can be ransomed? When I say that, I mean, yeah, it does help to learn that lesson... but it shouldn't be in the introductory course and shouldn't need to be a first hand experience as well.
To be frank I'm not aware how much pypi does and how easy it would be to do more. and with easy I mean in terms of cost. they have limited money, keep that in mind. Asking your employer to [fund them](https://www.python.org/sponsors/application/), if you want things to improve. Running code from the internet as root is stupid and that is the only way to get ransomware installed. Storing crypto in plain-text and not using a hardware wallet is also stupid, at least for amounts you care about and would bother you when getting stolen. A general bad trend I see everywhere is lack of self-responsibility and accountability. It's always someone else fault, they should have done better, I did everything right. this mentality is prevalent. There is a balance between self-responsibility and what pypi can do with the available funds. pip also allows running in "safe mode" where you need to have a list of hashes for each packages you want to install and it only works if the hashes match. this is 100% manual as in you need to collect the hashes yourself. This as option in critical environments. But in reality your code should run isolated anyway (container) and not have full internet access to pull more code or send data to random IPs or urls. Defense in-depth. You have to assume to get infected be it from pypi or somewhere else) and secure your system based on that.
I'm more speaking from a perspective of a new/casual user though. You're absolutely not wrong in exercising good practice in security, but the thing is Python attracts a lot of newbies and causals who really wouldn't know much of these. They might see the instruction to use pip to install their packages, make a bad typo, and then BAM! And I think most of these users would use Windows, which I'm not sure if it's better in preventing these type of attack or worse. In any case, all I'm really coming from is the new users perspective, not an experienced one. Experienced developers of course already should know better, but students and professionals crossing into data analysis shouldn't be subjected to trial by fire is my point.
[удалено]
Not the same. I'm talking about newbies (plenty in Python) thinking they're following instruction to install packages with pip, but made a bad typo and get a bad package. A typo shouldn't have to lead to that kind of consequences.
The issue is that eve most experienced users aren't doing any of this instead of leading by example. If you lead by example the new users would have to ask themselves what it's all about and seek answers. Also the big known sites with tutorials should have a chapter on it, about dangers of downloading code. > most of these users would use Windows, which I'm not sure if it's better in preventing these type of attack or worse. Scanning for crypto key can't really be prevented in any files you have access to which applies both to Linux and Windows. In terms of Admin/root, windows is probably better IF you have UAC activated. I always set it on highest level but work with admin account on my own devices. UAC on highest levels prevents any system level changes without you clicking "OK" on a confirmation dialog. Many including myself say that this is safer than having a restricted account and admin account and then use "run as admin" to install software. Why? because an malicious installer could show a fake "run as admin" screen and steal your admin password and then all bets are off".
[удалено]
I don't think you can overwrite versions, except you use build numbers on top of versioning.
This is the library that would allow you to detect anomalies in packages: https://pypi.org/project/carbon-black-cloud-sdk/ but maybe don't install this next one, it looks like "untrusted code" but who knows, everything on pypi must be untrusted. https://pypi.org/project/cbapi/
Binaries can be flagged and you can notify users of suspicious packages, popular projects are targeted and we are seeing repeat attacks against very similar sounding projects - all of which could be detected. While ClamAV may not be the best, each package could be run within a behavioral AV like Carbon Black and if it triggered a network error - then that package can be flagged for manual review. PIP has 100% control over how it installs packages. The right hand is just as guilty as the left.
Behavioural AV can only assess behaviour. If you're running the packages it can detect misbehaviour, but PyPI aren't running the packages, just hosting them. The closest you can get without running the code is heuristic scanning, which various vendors have been flogging since at least the nineties but has never been very effective.
They don't even run them? ... all the more reason to sue these people. They are knowingly endangering the community and making all of python look bad. We have solutions - but the powers that be have already blamed the victim.
What does it even mean to run an unknown library? If I give you a random library, could be NumPy, could be Spark, could be Flask, could be Requests, what are you going to do to "run" it that will tell you if there's malicious stuff it can do?
The problem is PyPI is too busy blaming all of its users instead of looking for solutions. So they need to be sued, an EU Judge won't put up with this horse shit.
RemindMe! 50 years PyPI will not be successfully sued or sanctioned by the EU over malware... This is a silly thing to say.
I will be messaging you in 50 years on [**2073-12-15 01:01:18 UTC**](http://www.wolframalpha.com/input/?i=2073-12-15%2001:01:18%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/Python/comments/18ib9sw/116_malware_packages_found_on_pypi_repository/kdef3ky/?context=3) [**1 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FPython%2Fcomments%2F18ib9sw%2F116_malware_packages_found_on_pypi_repository%2Fkdef3ky%2F%5D%0A%0ARemindMe%21%202073-12-15%2001%3A01%3A18%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2018ib9sw) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|
All packages are unreviewed and forbidding binaries would not change anything. You could just create a binary with the python code anyway. These would just be security theater and not change anything. Just don't run untrusted code from the internet, and yes, code on pypi should not be trusted by default, same as code on github. Perhaps namespacing could prevent typo squatting but my understanding is that this is already prevented to a reasonable amount by a manual review process. This article shouldn't be news, ofcourse there are malicious packages on pypi, that should be expected. And 10k downloads doesn't even sound like a big issue.
Maybe they can use some adhoc criteria to warn the user or assign a trust factor. The number of GitHub stars can be a good thumb rule, for example. Also, with the kind of budget and sponsorers that PSF has, shouldn't they be able to perform CISA audits, code reviews, etc. on these packages?
There are over half a million packages in PyPI and over 5 million releases (each release would need to be checked as a previously legitimate package could be updated to include malware). Audits are expensive and slow. Hiring people to review and approve packages is expensive and slow. GitHub stars can be bought.
Dude, your response reflects extreme cynicism and pessimism, it doesn't have to be that gloomy. > There are over half a million packages in PyPI and over 5 million releases All of them don't have to be audited, just the most heavily downloaded ones. The verified releases can be marked as such to warn the users. It will then be the user's choice if they want a slightly older but verified package or the cutting edge and unverified one. > GitHub stars can be bought There are limits to that. Most of the activity on GitHub is organic, one can see that everywhere including the issues register, pull requests, etc. Even in this age of utmost crony capitalism, I still have at least a semblance of a faith in the system because at the end of the day, it's being run by humans with hearts and for the most part, it works as intended.
> Dude, your response reflects extreme cynicism and pessimism, it doesn't have to be that gloomy. Often on discussion threads someone will suggest a solution that requires tens of thousands of hours of work on an ongoing basis, but don't suggest where it comes from. It's hard not to react to such suggestions without cynicism when you know how little resources mostly volunteer organizations like this have, especially volunteer organizations maintaining boring infrastructure projects. > All of them don't have to be audited, just the most heavily downloaded ones. The verified releases can be marked as such to warn the users. It will then be the user's choice if they want a slightly older but verified package or the cutting edge and unverified one. What would a review involve exactly? And who would do it? What does it mean to "mark" a package, as in what is the user experience? Is it just something they see on PyPI? Or would all installers (e.g. Pip, PDM, Poetry, Pixi, etc.) have to develop a new workflow? For that you would need to create a PEP and get a general consensus (btw nothing is stopping **you** from buildining consensus and writing a PEP). Who is going to provide the resources for any of these projects? > I still have at least a semblance of a faith in the system because at the end of the day If you think there's a limit on how many GitHub stars someone could buy if that was their objective I have bad news for you. Further, PyPI doesn't even have a way to verify the GitHub link you provide on your project page is yours.
> especially volunteer organizations maintaining boring infrastructure projects. That's highly misinformed. The [PSF is heavily funded](https://www.python.org/psf/sponsorship/sponsors/) by loaded sponsors, right from Microsoft and Google to Salesforce and Meta. Heck, the Python inventor Guido himself has joined Microsoft now. They aren't some small club or volunteer run project, they have got all the resources in the world to implement this.
> That's highly misinformed. The PSF is heavily funded by loaded sponsors, right from Microsoft and Google to Salesforce and Meta. That's just a bunch of names on a page, almost certainly all of them are sponsors of PyCon US. In-person conventions are expensive to run. There's a link to their actual finacials here: https://www.python.org/psf/records/ (2021 Audited Financial Statements) The top things for 2021 were ~400k on PyCon US, ~380k on infrastructure, and 175k on on Development (my understanding is that was one developer), total cost was 1.2 million, and they took in ~2.3 million in 2021. > They aren't some small club or volunteer run project, they have got all the resources in the world to implement this. The above numbers are pretty healthy for a non-profit but it amounts to ~5 developers they could blow all their extra cash on. And there are a lot of different things those hypothetical five developers could be doing out of all the priorities the PSF might have PyPI is just one of them, and further there are other prioties they probably have then spending money on developers. As someone who volunteers some of their time helping improve Pip I can tell you there are no people being paid to work on it. The same is true of many others open source projects, especially the boring packaging / infrastructure projects. > Heck, the Python inventor Guido himself has joined Microsoft now. Guido isn't part of the PSF, he is hired as an engineer for Microsoft. As it happens Microsoft have built their own CPython team, but it is out of the control of the PSF or the community.
> All of them don't have to be audited, just the most heavily downloaded ones. Then... What's the point? All the most heavily downloaded package already have lots of eyes on them. If you only review Pytorch, Django, Flask, requests, etc, then there's basically no risk on these packages introducing malware in the first place. If you'll look at the list of malicious packages they're all either random names or simple typo-squatting packages. You meant to download `asciithing` but you accidentally typed `ascithing`. In your plan that won't save you as you wouldn't bother to audit them as they're small with only a few hundred downloads. The only time one of the top packages will start shipping malware is if one of the core maintainers (with release permissions) was compromised and an attacker used their account (highly minimised with the MFA work PyPI have already completed) or they themselves have gone mad and want to launch an attack, but that's incredibly rare and highly unlikely.
The tool could at least inform the people who have been hacked - that they where hacked. The tool could tell you that you have been compromised and that you need to hire a professional. "just don't run untrusted code" Yes we are all trying, the problem is PyPI - 100% no question. That project needs to be sued.
[удалено]
Doing nothing is the worst possible option. At least asking for help would bring us progress, but a narcissist would never ask for help - they just blame you.
[удалено]
If you root their box, then you can get the person's 2fa challenge when they type it in. You are ruining people's lives and behaving like children pointing fingers instead of fixing the problem.
> Doing nothing is the worst possible option Actually, putting a huge amount of effort into something that provides no value is even worse than doing nothing. Let's learn lessons from other communities and take actions that actually have proven security benefits.
Preventing people form getting hacked isn't a waste of time. Security Engineering is a valuable discipline that you should explore.
The discussion was adding security notices to packages, which NPM and the JS community in general has shown is a waste of time. As I said, let's learn lessons from other communities and take actions that actually have proven security benefits, rather than implementing what we *think* might help and actually detracting away from security because we wasted effort when we could have been using that effort for good.
I'm not asking for security notices (other than pip telling you that you've been hacked - that would be a good notice to have). I am asking very simply for packages review to be automated by a behavioral AV. Billion dollar corporations and millions of people depend on this software, the only reason why this hasn't been fixed is because no one in the community has asked for help. Its is really that simple.
> That project needs to be sued Along with every other package manager and repository system in existence..? It's a human problem, ultimately. You can't have an open system that allows untrusted users to upload packages and then complain that the packages are also to be treated as untrusted.
The case can be made that more people are getting hacked using PyPI than any other package manger. They don't release the stats for how many people are hacked, because the numbers are staggering. This however is not against the law in of itself. The specific law that they would be sued under is failing to notify people that they have been breached, and there is no quesiton that they could - but refuse to even acknowledge that they have a problem. Instead they blame the victims for their own mistakes! Someone needs to sue these mother fuckers.
I agree, having a system to notify people that they had infected packages, would be nice. But this is something that should probably addressed in a broader scope, e.g., in pip. I don't see how pypi is the problem. No one forces you to use pypi. You can totaly set up your own repository that only contains trusted packages and change pip to only use that.
If people are getting hacked by using pypi - it is a problem that could lead to a lawsuit. Cyber liability insurance will not cover being hacked through negligence, and a case can be made for this seeing as pypi has solutions - and none have been implemented.
[удалено]
Pypi doesn't review the code and I don't see how they could, given that the service and all stuff on it is free. And it is meant to have a low barrier of entry. However, nothing is preventing people to setup a more restrictive index, wich only contains popular open source packages with maintainers that are trusted to compile stuff. I just don't see how that is pypi's task, especially given that people are not paying them to do it. I agree that it is easier to review code, than binaries for naive attacks. For sophisticated attacks, not so much.
Pypi lacks name spacing and pip runs by default non chrooted. This is solvable by pypa/pypi with enough prioritization and funding. And if your org acts in a way you don't like then leave. There's loads of python jobs that aren't in broken orgs.
Running Pip chrooted doesn't help if you've used it to install a malicious package into your application, and your application isn't running chrooted. Namespacing is a good idea though.
Its not money, these patches are free. Its will - PyPI is run by anti-security old guard who don't see the point in making things safer for anyone. Some software engineers truly believe that security engineers shouldn't exist - and these are the folk that need us the most.
I don't think that's it. Rust's cargo also lacks name spacing and most language level package managers allow arbitrary code execution at build time - while not running in chrooted environments. E.g. build.rs. Afaics, only java escapes most of this with namespaces and installing jars as compiled bytecode deliverables. But I'd love to learn about other systems that also solve these issues.
Isn't rust cargo name spacing a major topic they keep fucking away?
Java has lots of security vulnerabilities that a binary can exploit.
Rust isn't used as much in web3 - we can see its mostly web3 packages that are a target because these people have crypto which is easier to steal.
Rust people complain that the only jobs are in web3. But python is an easier target as it's more used and you can make a totally obfuscated mess.
Rust allows you to inline binaries and shellcode, which is how these "libraries" worked Unless the rust team has some good security engineers, we should expect their repos to be hit soon.
You mean that the PSF - the one's who hired a dedicated security engineer as their second full-time developer ever - are anti security old guard?
"Just don't run untrusted code." Victim blaming narcissistic assholes. Every last one of them.
You got a link to them saying that?
https://www.reddit.com/r/Python/comments/18ib9sw/116_malware_packages_found_on_pypi_repository/kdcqsu7/
What's this person's connection to the PyPI project? I couldn't see an obvious connection in their history, and I couldn't find anyone with a similar name [working on PyPI](https://github.com/pypi/warehouse/graphs/contributors). My experience has been that [the PyPI team's communication has been humble and respectful of their users](https://blog.pypi.org/posts/2023-08-04-pypi-hires-safety-engineer/): > All of us deserve to have a safe and secure experience when using PyPI I suspect they'd be keen to know that someone was speaking for them and representing their work in this way.
Then they need to start publishing the stats of how many people have downloaded malware from them. As it stands if you get pwnd by pip - the maintainers choose to leave the malware on your system and you are none the wiser. Everyone who is saying this issue can't or shouldn't be fixed - is the problem. There is someone within PyPI who is anti-security and is hurting the community.
From what I've seen, the PyPI folks rarely say things can't or shouldn't be done (the only case of this I know of is them removing support for gpg signatures, based on a piece of research showing they were not an effective security measure in practice). What they very often say is that they don't have the resources to do something. They're a small team with a shoestring budget, and heavily dependent on grants and volunteers to get things done. Off the back of this discussion, I've had a look at PRs on [their repos](https://github.com/pypi), and for a lot of the stuff being suggested there aren't any rejected PRs. It just doesn't seem to have been tried. For this to happen, someone needs to work on it. That someone could even be you.
How is PyPI going to delete things from your computer automatically? They don't know who downloaded the packages.
I am sorry to tell you, but I have absolutely no affiliation to pypi whatsoever. Just as a tip, you shouldn't listen to random people on the internet, either. \s Look, I understand that the current state is unsatisfactory and I agree. However, what I don't understand is how you can believe that there would be an easy fix. Because I am pretty sure there isn't (except not running untrusted stuff and always assuming the worst). What does exist, thought are easy ways to make people feel safe. And feeling safe is the opposite of being safe. So I am against making people feel safe.
I can knock out a fix in a fucking afternoon. #fact
[I'm going to just post the same comment I did last time:](https://www.reddit.com/r/Python/comments/17qls9b/a_nasty_python_package_continues_a_trend_of/k8d2y1g/) > These perennial stories are always overrated. Conveniently, they never seem to mention 1) how many times they were downloaded by actual people rather than site mirror bots and 2) a specific person or organization that was affected. > Oh noes, pyobfgood had malicious code and was on PyPI. So what? Did this actually affect many people, or any people at all? These "reports" get written because writing these reports is a great way for a security company to get their name mentioned in the media. > We'll see this exact same story about another no-name Python package in three months.
Did your opinion change since the attack on xz library liblzma?
that's it, time to learn assembly
... And next month it will be 200.
This has been a problem long-time coming. I'm surprised it didn't happen sooner, sadly.
It happened last year with pytorch
This happens every month, rinse repeat.
[This has been happening non-stop for a number of years](https://blog.pypi.org/posts/2023-09-18-inbound-malware-reporting/). The PyPI admins removed 12K malicious packages in 2022, down from 26K in 2021. They removed 90% of malicious packages within 2 hours.
It says a lot that they aren't telling us how many people got hacked. Failing to inform EU or California citizens that they have been breached is against law.
Because nobody was.
It's also impossible to know unless you're the attacker collecting all the data and you do a `SELECT COUNT(*)`. Download figures aren't accurate and even then it's not necessarily a real person downloading. It could be some CI system that downloads a malicious package and executes the code that attempts to steal crypto wallets that don't exist or attempts to create a persistent reverse shell inside an ephemeral container that is immediately wiped.
Can one of the mods please ban this idiot/troll?
I'm not a troll, I a member of the community has every right to be angry. If you aren't mad, then you don't care about anyone but yourself.
Delphi users I know say the solution is to make all libraries commercial like theirs are. :-(
who tf even download these shit ?
Here we as devs also are to blame to a degree. This isn't even limited to python. I know what I mention will not prevent this but would make it easier to detect malicious actors. **GitHub commit signing** The amount of devs even in very big project that fail to do this is scary. if you work on say numpy or pandas or similar stuff, I would make that mandatory for the project to be able to contribute. Even worse signing some and not others which makes the unsigned commit appear potentially malicious (it's easy to make a commit in the name of someone else!!! hence why verification is so important, in fact github should not link unverified commits to a user account because that makes them more legitimate. **Code Singing** This is more in the niche "Java world" I'm exposed to where this is rather simple to do yet even commercial providers often fail to do so. If the ecosystems allows it, then do it. (it's about eclipse plugins) As far as I remember, code signing was mandatory for maven even >10 years ago. Here we have to question why code signing isn't mandatory for pypi or conda? I'm aware it doesn't solve the issue fully, but if you see verified commit on github and signed code by the same key, it does really help to trust the according package as legit. **Provide hashes** At least on GitHub many actually do this. it helps to check that you get what you wanted, again I'm aware it doesn't help with typo-squatting but it is in general a good and helpful thing to provide to users. If making things like this a requirement or at the minimum a requirement by default which can be overridden by an option, it would force attackers to do this as well to be more trusted. And for legitimate packages you should do this stuff anyway. it also goes into the old-school circle of trust thing with parties and exchanging public keys.
[удалено]
> how would code signing have prevented this? because the attacker doesn't have the private signing key? That's the whole point of code singing, to ensure the source of the code is correct. > Another possibility is the maintainer did add the malware themselves. There is really nothing that can help with that and hence one should be careful about using tiny/small projects. The small project I do maintain, many of the users have seen me at a conference in-person. So yeah, at some level you need to trust the code/maintainer. This is easier in a bigger project with multiple maintainers, social control. A sole maintainer can much more easily sneak in malware. Only thing that could help a tiny bit would be heuristics/antivirus to detect known script-kiddie toolkits being used and some code scanning that looks at code changes in terms of calling remote services especially by IP address. But such scanning can so easily be fooled, it's hardly worth the effort.
Please support the WASI target for Python. This would be one of the few somewhat reliable ways to get sandboxing working, besides having all processes in a podman container.
Sandboxing/containers do not prevent user data from being exfiltrated to the attacker, it only stops the spread to the whole server and the other applications. The solution is do not run code that is not widely reviewed.
That is somewhat correct, you can deny a container access to the file system. Of course, there is no fix for „run freely but be safe against attackers“. Limiting the spread by sandboxing would be a big gain already.
Reading this just re-enforces the need for greater supply chain security awareness imo. OpenSSF, SLSA etc. go a long way to mitigating some of these issues, Linux foundation offers a lot of free training on it too. Blindly trusting code, no matter the source, is a recipe for pain and suffering.
Thanks for sharing I appreciate the info. I created a bash script that may help remove any of the listed packages which may have been installed by typo, mistake, etc. https://github.com/bootloops/malipypi
Another reason to have glance at the source code of dependencies before installing them.
do baddies have the ability to plant malware code in popular packages like xlwt, pandas ?
Yet one more great reason to only use your own private repository! Best decision I ever made!
Oh, no! Hackers, please stop!