Checking a competitor’s price once is easy. Checking 2,000 products across four countries every morning is a different job. Do it from one connection and you’ll either get blocked, or worse, get answers that look fine but are wrong.
This guide covers how to collect prices that match what real shoppers see, how to choose proxies for different kinds of shops, and how to catch bad data before it ends up in a pricing decision.
Why price tracking needs proxies
There are two reasons, and the second one matters more.
Blocking. Retailers watch for visitors who load hundreds of product pages in a row, because that’s exactly what competitors do. Spreading requests over many residential IPs keeps your checks looking like ordinary shoppers.
Accuracy. The price a shop shows can depend on where the visitor is. Currency, taxes, shipping costs, regional promotions and stock all vary by location, and some shops test different prices on different visitors. A price collected from the wrong country, or from a connection the site has flagged, may not be the price your customers see.
A failed check is obvious. A wrong price that looks right is the one that costs you money.
What to collect for each product
The price on its own rarely tells the full story. For each check, save:
- the current price, and the “was” price if there’s a discount
- currency, and whether tax is included
- stock status, or the delivery date if it’s shown instead
- shipping cost to the area you’re checking from
- the seller, on marketplaces where several sellers offer the same item
- any promotion, coupon or bundle shown next to the price
- the country or city you checked from, and the date and time
The last item is easy to skip and painful to miss. Without it, you can’t tell a real price change from a check that ran in the wrong place.
Check from where your shoppers are
Set the proxy country to the market you’re comparing. For the United States, state-level targeting helps with sales tax and delivery options. For grocery, food delivery and anything with local stock, go down to the city.
Travel is the extreme case. Flight and hotel prices can change with the country you book from, so if you compare fares, check each route from each market you sell in.
Rotating or sticky sessions
For plain product pages, use rotating sessions so every page comes from a new IP. Some shops only show a final price after you choose a delivery postcode, pick a size, or add the item to a basket. For those, use a sticky session so the whole sequence happens on one IP, the way a real shopper’s would. Sticky IPs on our network can be kept for up to two hours.
Choosing a proxy type
- Large retailers, marketplaces, airlines and hotel sites usually need residential proxies. They invest heavily in spotting automated visitors.
- Smaller online shops, B2B catalogs and supplier price lists often work fine with datacenter proxies at about half the cost per GB.
- Prices only shown in a mobile app or mobile site are best checked through mobile proxies.
Keeping the data trustworthy
Price data feeds pricing decisions, so a few automatic checks are worth the effort. Flag a result for a second look when:
- the price is zero, missing, or many times higher or lower than last time
- the currency doesn’t match the country you checked from
- the page is much smaller than usual, which often means a block page or an empty template
- a product suddenly shows as out of stock everywhere at once
Before you act on a big change, check the same product again from a different IP in the same country. If both agree, it’s real.
How often to check, and what it costs
Match how often you check to how quickly prices move in that category:
- flights and hotels: several times a day
- electronics and popular marketplace items: daily
- fashion and homeware: a few times a week
- groceries and slow-moving catalogs: weekly
Within a category, check your bestsellers more often than the long tail. That alone can halve your volume.
For a rough budget, take 2,000 products in 4 countries, checked daily. That’s 240,000 page loads a month. At around 200 KB a page, that’s about 48 GB, which works out to roughly $81.12 a month on residential proxies or $42.72 on datacenter proxies. Most teams end up with a mix: datacenter for the sites that allow it, residential for the ones that don’t.
Staying fair
Public prices are fair game for comparison, and checking them is everyday work for retailers, brands and resellers. Keep to prices anyone can see, leave members-only prices alone unless you have an account that allows it, and ease off during big sale events when the shop is already under heavy load. Brands checking that resellers keep to agreed minimum prices can use exactly the same approach.
Questions and answers
- Can I track prices on large marketplaces with proxies?
- Yes, that is one of the most common uses. Large marketplaces are strict about automated visitors, so use residential proxies with rotating sessions, keep a steady pace, and target the country whose prices you want to see.
- How often should I check competitor prices?
- It depends on the category. Travel prices can change several times a day, electronics daily, and groceries or slow-moving catalogs weekly. Check your most important products more often than the rest.
- Why are the prices I collect different from the ones in my browser?
- Prices can change with the visitor's country, region, currency, whether they are signed in, and sometimes with price tests the shop is running. Make sure your proxy location matches the market you want, and compare like with like.
- What is the cheapest way to monitor prices?
- Use datacenter proxies for every site that allows them, and residential proxies only where you need them. Download just the product page without images, and check slow-moving products less often.