Astro Hacker News - PyInfra 3.8.0 Is Out

ssddanbrown |next [-]

I've been using PyInfra for a while, albeit just for simple automation (Updating systems, checking certain stats) and I'm a big fan. Compared to Ansible, I found the docs, syntax and usage patterns much easier to get on with. Might just be a preference thing, but I always had trouble going through the Ansible docs.

Ran into some bugs, like one machine that seems to cause errors and mess up the output on restart, although that looks like it might have been addressed in this release.

If it helps, I put together a video when initially exploring PyInfra: https://www.youtube.com/watch?v=S-_0RiFnKEs

wowi42 |root |parent [-]

Glad it clicked. The Ansible vs PyInfra docs gap isn't really preference, YAML plus Jinja plus a custom DSL is just more cognitive load than plain Python with type hints. Once you can grep the source and read it like normal code, going back feels rough. On the restart bug: if it resurfaces, an issue on GitHub with the OS, connector (ssh/local/docker), and raw output would help a lot. The 3.x line cleaned up a bunch around connection handling and output buffering, so there's a decent chance it's already fixed. Thanks for the video, will watch. Hands-on intro content is exactly what the project needs more of.

wowi42 |next |previous [-]

Disclosure: PyInfra core contributor here.

We just shipped 3.8.0.

PyInfra is an agentless infrastructure automation tool. Same job description as Ansible, Salt, Chef. SSH into hosts, describe desired state, it diffs and converges. No agent, no central server, no daemon.

The difference: your "playbook" is just Python. Not Python cosplaying as YAML. Not Jinja smuggled inside YAML inside a Helm chart inside a Kustomize overlay. Actual Python:

    from pyinfra.operations import apt, files, server

    apt.packages(packages=["nginx"], update=True)
    files.template(src="nginx.conf.j2", dest="/etc/nginx/nginx.conf")
    server.service(service="nginx", running=True, enabled=True)

Idempotent operations. Facts gathered from hosts, branched on with normal `if` statements. Real loops, real imports, a real debugger, real type hints. Your editor autocompletes arguments because, brace yourself, they are just function signatures.

About YAML. Wonderful format. For about eleven minutes. Then someone needs an `if`, and you have `{% if %}` inside a string inside a list inside a map. Then someone types `no` as a country code for Norway and it ships to prod as `False`. Then someone indents with a tab and the parser dies without saying where. Congratulations, you reinvented a programming language. Badly. The honest move is to admit you wanted code, then write code.

PyInfra skips the eleven good minutes and goes straight to code.

Release notes in the link. Happy to answer questions.

Infrastructure as Code, not infrastructure as YAML.

zahlman |root |parent [-]

As a heads-up, your comments here were flagged. I think some people must have thought your (current) writing style rather LLM-ish.

wowi42 |root |parent [-]

Ah shit...

V__ |next |previous [-]

Has anyone used this and ansible and is able to give a short comparison with likes and dislikes?

matthiaswh |root |parent |next [-]

I switched from Ansible to Pyinfra for my homelab, and continue to use Ansible at work.

The biggest difference is that Pyinfra is simply Python code. It's incredibly easy to control the system in whatever manner you need to. You can probably do the same thing in Ansible, but it's never quite as obvious how to do it. This also means it's much more clear where and why things work the way they do in Pyinfra, where in Ansible I end up digging through numerous role files to try to find where some variable gets injected.

SteveNuts |root |parent [-]

The worst part of Ansible is data manipulation, what would be an easy dictionary operation in Python is a huge mix of lookups in Ansible.

Incredibly frustrating that the data you want is right there but you can't easily grab it.

gchamonlive |root |parent [-]

If Jinja templating for data manipulation gets too complex or inconvenient, you can create your own module in ansible and use python code for data manipulation. But at this point you are better served with plain python which I think is where pyinfra should shine. I want to take a look though at how hard it is to implement your own module for it.

matthiaswh |root |parent [-]

If you're doing data manipulation with server state, you would probably want to create a Fact[0]. This is a straightforward Python class.

If you're doing data manipulation locally you would simply write Python code.

Operations[1] are Python functions which execute (yield) commands which will be run on hosts.

That's the gist of what it takes to write custom modules for Pyinfra.

[0] https://docs.pyinfra.com/en/3.x/api/facts.html [1] https://docs.pyinfra.com/en/3.x/api/operations.html

Boxxed |root |parent |next |previous [-]

Pyinfra is what ansible should have been. It's straight python rather than a janky mix of yaml, templates, and bolted-on control flow primitives.

polski-g |root |parent [-]

There is this: https://github.com/seantis/suitable

But the main guy who developed it at that company left, so no idea on its longevity.

wg0 |root |parent |next |previous [-]

I used ansible for building simple logging appliance (something like Elasticsearch + Dashboards + other tooling) and I found it very difficult to reason with specifically python code snippets within YAML.

Switched to Pyinfra and the difference is day and night. You write python code you can organise your stuff into functions, classes and whatever you like and then instantiate them as you like. Highly reusable configuration.

You have full pwoer such as you can call boto to fetch the list of servers to target, filter base on tags and what not. Only sky is the limit because it is NOT a DSL (or YAML) rather full blow real python.

hylaride |root |parent |next |previous [-]

At a previous job we used it to test our ansible playbooks via molecule, which were part of a CI/CD pipeline to create AWS AMIs.

It worked well and was nicer to deal with than test kitchen for testing UNIXy things (is service running and/or enabled, does file have right permissions, does file include $TEXT, etc). It was very useful for us during big linux upgrades, such as when ubuntu went from upstart to systemd. It can also be good at capturing edge cases with brittle outcomes (especially as ansible went through enormous changes after the red hat acquisition).

Dislikes? I had to fight with pyenvs a bit..

gegtik |root |parent [-]

was this before uv? i feel like my pyenv struggles basically ceased once I started using it

hylaride |root |parent [-]

I used it between 20016-2023 and since we were not a python shop, I never used any other package managers. It was never an issue with CI/CD pipeline, but iterating locally was always a fight to getting molecule to pick up the right pyenv. It got better towards the end, though.

Honestly the bigger issue was testing x86 docker images on an arm mac, as molecule didn't cleanly support cross platform images and we did pull in x86 binaries for our playbooks (by the end of my time at said company, I was also directly managed by product managers who didn't care about tech debt and I couldn't deal with the otherwise desirable idea to move our compute to ARM - a rant for another day). This may also be fixed now.

haolez |root |parent |previous [-]

Ansible includes modules to handle cloud resources as well, such as AWS Lambda.

coreylane |next |previous [-]

I used ansible for years and pyinfra is very approachable since it has similar concepts, like inventories, common operations like files.put, server.shell, loving it so far, and it is quite fast

eb0la |next |previous [-]

This reminds me of Nortel Command Console back in 2000-2005!

I worked for a telco company that had a lot of Nortel Passport devices (does anyone know what Frame Relay is?). We started changing the network from Nortel to Cisco. Cisco used telnet (later SSH), but Nortel people were extremelly reluctant to switch.

Turns out the Nortel network managment system (nortel nms) had a very interesting feature: you could open the command console to connect to one of the passport devices... or you could connect to a device group (or all the network) and run the same command in all devices.

This was great for auditing which version had every single device in the network... or for changing access-lists globally.

cpach |root |parent [-]

We had lectures on Frame Relay and stuff like that in uni, but I’ve never ever touched that stuff (:

bestony |next |previous [-]

This looks great! pyinfra will integrate better with my other code, and installing it with uv fits my workflow better. Thanks for the post. I'll give it a try. I think some of my Caprover initialization tasks could also be handled by pyinfra.

hathym |next |previous [-]

i tried ansible before and hated it, this idea is genius.

sgarland |next |previous [-]

Never heard of this before. In looking through docs, honestly it looks like Ansible, but for people who don’t know Ansible, and with way more footguns. The fact that you can import any existing Python library means you’re now relying on those libraries to not introduce bugs, or throw an exception in the middle of an operation, etc.

I despise YAML, but I can appreciate that it makes it harder to introduce imperative logic, and it forces you to stay on the paved path - which is very well-tested.

hacker161 |next |previous [-]

See lots of comparisons to Ansible but Chef/puppet (both of which have agent-less modes) in Python instead of Ruby is what immediately came to mind. I guess Salt as well technically.

appplication |next |previous [-]

The is cool, thank you for sharing. I was just thinking about onboarding to ansible since I’ve just been following a manual checklist of commands for my remote server but based on positive feedback here I’ll probs oh give this a shot. Only downside is I imagine LLMs are probably a little more proficient at ansible just due to volume of training data.

coreylane |root |parent [-]

I never depend on a models built-in training when using third-party libraries. Providing tons of additional context to the model like a skill, example repos, or context7 snippets that I manually curate is more effort up-front and takes longer, but the results are worth it.

Stuff I threw into the inputs before working with pyinfra

https://github.com/pyinfra-dev/pyinfra-examples

https://context7.com/websites/pyinfra

odie5533 |next |previous [-]

Does it have an equivalent to konstruktoid's hardening Ansible playbook?

wowi42 |root |parent [-]

we could put it on our roadmap of examples :-)

ktm5j |next |previous [-]

This seems cool, I'd particularly be interested if their 10x faster than Ansible claims pan out. Has anyone here used PyInfra? If so what's your experience been like?

eurekin |root |parent |next [-]

On my homelab. It really feels like a dream come true for my usecase. No more puppet agents. No more declarative syntax, that you have to work around to do basic imperative ways. Or use a module, that stopped being maintained 3 years ago. Just plop a file here and there through ssh.

alanwreath |root |parent [-]

Same here, my home lab is all pyinfra. I’m not sure if it’s my previous experience with ansible that made it simple for me or just the relative size of my home lab compared to larger companies where I’ve used ansible - but it seemed much easier to me and easier to follow.

e12e |root |parent |previous [-]

I wonder if this is 10x faster than Ansible with ssh multiplexing or not?

https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing

mark_l_watson |next |previous [-]

That would have been very useful to me, before I retired! That said, I only run the Hermes Agent on leased VPSs and PyInfra might be a cool and easy to access Hermes - I need to think about that.

dist-epoch |root |parent [-]

I tried something like that, using PyInfra to setup VMs for agent. But gave up, too much complexity for too little gain. Just ask the agent to create a small install script.

pbronez |next |previous [-]

How does this compare to Salt Stack?

“Built on Python, Salt is an event-driven automation tool and framework to deploy, configure, and manage complex IT systems. Use Salt to automate common infrastructure administration tasks and ensure that all the components of your infrastructure are operating in a consistent desired state.”

https://docs.saltproject.io/en/latest/topics/about_salt_proj...

bijowo1676 |root |parent [-]

salt is heavy: it has client, server, agent. it equires installation, consumes resources, etc.

pyinfra is just python that gets transpiled into ssh commands

subhobroto |next |previous [-]

Congrats on shipping 3.8.0!

If you're a software engineer who wants to setup and maintain infrastructure, give PyInfra and Pulumi a go!

Huge fan of PyInfra. For my homelab, I use Pulumi with Python and PyInfra to build fully declarative intent based infrastructure. You can use actual software engineering principles like composition, inheritance, DI to setup and wire your infrastructure and services. One of the benefits of this is your infrastructure and services are now self documenting (have them write out a mermaid diagram!) and easily testable using pytest (from cheap unit tests to extensive integration tests (I use Incus)).

Instead of Pulumi, I originally used Terraform CDK with Python before CDK got IBM'd. The migration to Pulumi was refreshingly painless. My original reason for not choosing Pulumi was the crippled state of the open source, self hosted backend support a decade ago but it looks like that is now way more mature and less crippled.

PyInfra is a breath of fresh air compared to Ansible - its not just fast, it's more Pythonic, so IDE features actually work, readable, maintainable, debuggable. I call it infrastructure for software engineers.

If anyone wants to use an AI agent to try out PyInfra - One issue I've faced is that PyInfra was rearchitected in v2 (and some more in v3?) but what belongs in v1 vs v2 vs v3 isn't very clear, so an AI agent could spend a lot of time writing v1 code, having it fail and iterate to v2 and then to v3.

The official site uses the version in the URL as the namespace but it seems like the SOTA AI agents don't pay much attention to that.

Maybe writing a llms.txt for PyInfra v2, or v3 would be an extremely useful task to help with onboarding newcomers?

---

The original post by the OP https://news.ycombinator.com/user?id=wowi42:

Disclosure: PyInfra core contributor here. We just shipped 3.8.0.

PyInfra is an agentless infrastructure automation tool. Same job description as Ansible, Salt, Chef. SSH into hosts, describe desired state, it diffs and converges. No agent, no central server, no daemon.

The difference: your "playbook" is just Python. Not Python cosplaying as YAML. Not Jinja smuggled inside YAML inside a Helm chart inside a Kustomize overlay. Actual Python:

    from pyinfra.operations import apt, files, server

    apt.packages(packages=["nginx"], update=True)
    files.template(src="nginx.conf.j2", dest="/etc/nginx/nginx.conf")
    server.service(service="nginx", running=True, enabled=True)

Idempotent operations. Facts gathered from hosts, branched on with normal `if` statements. Real loops, real imports, a real debugger, real type hints. Your editor autocompletes arguments because, brace yourself, they are just function signatures. About YAML. Wonderful format. For about eleven minutes. Then someone needs an `if`, and you have `{% if %}` inside a string inside a list inside a map. Then someone types `no` as a country code for Norway and it ships to prod as `False`. Then someone indents with a tab and the parser dies without saying where. Congratulations, you reinvented a programming language. Badly. The honest move is to admit you wanted code, then write code.

PyInfra skips the eleven good minutes and goes straight to code.

Release notes in the link. Happy to answer questions.

Infrastructure as Code, not infrastructure as YAML.

DandyDev |root |parent [-]

There is a PR open for llms.txt and llms-full.txt. We'll try to merge it soon!

Disclosure: another contributor here.

subhobroto |root |parent [-]

It's amazing to see more contributors!

TBH, I was worried a few years ago that there was basically just one (original) contributor. This now gives me added trust that I'm taking the right decision to lean heavily into it.

I hope more people start using pyInfra.

Thank You for your contribution and attention!

gandreani |previous [-]

There's a video!

I can't get over the fact of how suspicious he looks while doing it. And doesn't even cover his face. Crazyness

https://x.com/porqueTTarg/status/2047652413306277970 https://xcancel.com/porqueTTarg/status/2047652413306277970

alanwreath |root |parent [-]

This is spam - btw this is the first spam I have ever come across on hacker news

akshaykarthik |root |parent |next [-]

I think this was likely an attempted response to https://news.ycombinator.com/item?id=48008326

alanwreath |root |parent [-]

Yes - that’s got to be it.

electroly |root |parent |previous [-]

FWIW, if you turn on "showdead", there is a ton of spam on HN. The mods are just really good.

JSR_FDED |root |parent [-]

Showdead is quite a disheartening experience - there’s just so much LLM generated crap. The dead internet theory doesn’t feel as fringe as it once did.