
It’s 10:00 AM on a Tuesday. You’ve just finished your third cup of coffee, and you’re staring at a terminal screen that’s screaming a 405 Method Not Allowed error back at you.
We’ve all been there. You’re trying to build something cool—maybe a dashboard that pulls real-time pricing from a competitor’s site or a tool that automates your boring administrative tasks—and the communication between your code and the server feels like a bad long-distance relationship. You’re sending signals, but they’re being interpreted all wrong.
The culprit is usually a misunderstanding of HTTP methods.
At its core, the Hypertext Transfer Protocol (HTTP) is the language of the web. If the internet is a massive conversation, HTTP methods (often called "verbs") are the actions that define what we’re trying to do. Are we asking for information? Are we trying to change something? Are we throwing something away?
In this guide, we’re going to go beyond the textbook definitions. We’ll look at how these methods work in the real world, the common frustrations they cause, and how you can stop fighting with raw requests and start getting the structured data you actually need.
What Are HTTP Methods, and Why Should You Care?
Imagine walking into a library. If you walk up to the librarian and just say "The Great Gatsby," they don’t know if you want to borrow it, return it, donate a copy, or burn it (please don't burn books).
In web development, a URL (the "noun") identifies a resource, like /api/products/123. But the HTTP Method (the "verb") tells the server what action to perform on that resource.
If you use the wrong verb, the server gets confused. If you use the right verb but the server isn't set up to handle it, you get errors. Understanding these nuances is the difference between a "senior developer" move and a "weekend project" headache.
The "Big Four" You Use Every Day
While there are several methods defined in the HTTP specification, about 95% of your work will revolve around these four:
GET: "Give me this."
POST: "Create this."
PUT: "Replace this."
DELETE: "Remove this."
Let's dive into the technical details and the "gotchas" of each.
1. The GET Method: The Request for Information
The GET method is the workhorse of the web. Every time you type a URL into your browser and hit enter, you’re performing a GET request.
How it works technically:
A GET request should be safe and idempotent.
Safe: It doesn't change the state of the server. You can "GET" a page a thousand times, and the database shouldn't change.
Idempotent: Making the same request multiple times should produce the same result (assuming the data itself hasn't changed elsewhere).
import requests
# A simple GET request to fetch data
response = requests.get('https://api.example.com/v1/users/42')
if response.status_code == 200:
user_data = response.json()
print(f"User Name: {user_data['name']}")The Pain Point: We often try to use GET for things it wasn't meant for, like sending sensitive data (passwords or API keys) in the URL query parameters. Since GET requests are cached and stored in browser history, this is a massive security "no-no."
2. The POST Method: Creating Something New
If GET is for reading, POST is for writing. When you submit a signup form or upload a photo, you’re using POST.
Unlike GET, POST requests usually include a body. This is where the "payload" lives—the JSON or form data you’re sending to the server.
Why POST is different:
POST is neither safe nor idempotent. If you hit "Refresh" after submitting a credit card payment, and the browser asks "Are you sure you want to resubmit?", it's because it's a POST request. Resubmitting could mean creating a duplicate order.
// A POST request using the Fetch API
const newUser = {
username: 'dev_pro_99',
email: 'hello@example.com'
};
fetch('https://api.example.com/v1/users', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(newUser)
})
.then(response => response.json())
.then(data => console.log('User created:', data));3. PUT vs. PATCH: The Great Update Debate
This is where developers often trip up. Both are used to update existing resources, but they have very different philosophies.
PUT (Replace): Think of
PUTas "replace the entire thing." If youPUTa user object and forget to include theemailfield, the server might just wipe that email out because you provided a "complete" new version.PATCH (Modify):
PATCHis for partial updates. You’re telling the server, "Hey, just change theemailfield for this user; leave everything else alone."
Real-world advice: PUT is easier to implement but harder to use (because you have to send the whole object). PATCH is harder to implement on the backend but much more "bandwidth-friendly" for the client.
The Hidden Complexity: Why "Standard" HTTP is Often a Lie
In a perfect world, every website would have a beautifully documented REST API using these methods correctly. You’d send a GET to a URL and get back perfectly formatted JSON.
The reality? Most of the data you actually want—competitor prices, real estate listings, financial data—lives on websites that don't have an API.
The Traditional "Scraping" Nightmare
When there's no API, we resort to web scraping. We write scripts using libraries like BeautifulSoup or Selenium. We try to mimic a GET request from a browser, but then we’re faced with a wall of messy HTML.
The process usually looks like this:
Inspect the page source.
Find a specific
<div>with a class likeprice-tag__inner-23b.Write a regex or a CSS selector to grab the text.
Wait three days.
The website owner changes the CSS class, and your script breaks.
It’s a fragile, high-maintenance way to live. You spend more time fixing selectors than actually using the data. You aren't just dealing with HTTP methods anymore; you're dealing with the chaos of DOM structure.
A Better Way: Turning the Web into a Type-Safe API
What if you could skip the "scraping" phase entirely? What if you could treat any URL—even one without an official API—as a structured, type-safe endpoint?
This is where a tool like ManyPI changes the game. Instead of you manually handling the nuances of GET requests, headers, and fragile HTML parsing, ManyPI acts as a bridge. It turns any website into a structured API in seconds.
How it looks in practice:
Imagine you want to get product data from a retail site that doesn't have an API. Instead of writing 50 lines of Python to parse HTML, you can use ManyPI to get structured, schema-validated JSON.
curl -X POST
'https://app.manypi.com/api/scrape/YOUR_API_ENDPOINT_ID'
-H 'Authorization: Bearer YOUR_API_KEY'
-H 'Content-Type: application/json' The beauty here is that you're no longer worried about whether the server expects a specific user-agent or if the HTML structure changed. You define the type of data you want, and the service handles the HTTP complexity. It turns the "wild west" of the web into a predictable, type-safe environment.
HTTP Method Safety and Idempotency: A Quick Reference
To help you avoid those pesky 405 or 500 errors, here is a quick breakdown of how these methods should behave according to the spec:
Method | Safe? | Idempotent? | Body? | Purpose |
GET | Yes | Yes | No | Retrieve data |
POST | No | No | Yes | Create a new resource |
PUT | No | Yes | Yes | Replace a resource |
PATCH | No | No | Yes | Partially update a resource |
DELETE | No | Yes | No | Remove a resource |
What happens when things go wrong?
400 Bad Request: You sent a body (usually in
POSTorPUT), but the server didn't understand it. Check your JSON formatting!401 Unauthorized: You forgot your API key or Bearer token.
405 Method Not Allowed: You tried to
POSTto a URL that only acceptsGET.500 Internal Server Error: The server crashed. Often, this happens when you send a
PUTrequest but miss a mandatory field, and the backend isn't handling the null value correctly.
Best Practices for Developers
Use the right tool for the job: Don't use
GETto delete a record just because it's easier to test in a browser. It creates side effects that can lead to data loss if a search engine crawler accidentally hits that link.Version your requests: If you're building an API, use headers or URL paths (like
/v1/) so you can change your method logic without breaking everyone's code.Handle Status Codes gracefully: Don't just check for
200 OK. A201 Createdis the correct response for a successfulPOST, and204 No Contentis common for a successfulDELETE.Abstract away the "Muck": If you're spending more than an hour trying to scrape data from a site, you're losing money. Use services that provide structured data out of the box so you can focus on building your actual application logic.Final Thoughts
HTTP methods are the fundamental building blocks of the digital world. While they might seem like simple labels, understanding the difference between a PUT and a PATCH, or knowing when a POST is safer than a GET, is what makes your applications robust.
However, don't let the technicalities of "traditional" web communication slow you down. The goal isn't to be an expert at parsing messy HTML; the goal is to build something useful with the data.
Whether you're manually crafting a REST API or using ManyPI to bypass the headache of unstructured web data, remember: the best code is the code that is readable, maintainable, and—above all—functional.
Written by
Ole Mai
Founder / ManyPI

