1. 为什么选择 PHP 做 Amazon 爬虫？
表格
一句话：如果你本来就在写 Laravel，用 PHP 写爬虫等于「顺路」。
2. Amazon 页面结构 60 秒速览（2025-06 最新）
以 https://www.amazon.com/dp/B08N5WRWNW 为例：
表格
结论：95% 字段静态直取，无需上重型浏览器。
3. 环境准备：30 秒搭好最小可用环境
bash
# 创建项目 mkdir amz-php-crawler && cd amz-php-crawler composer init --name="demo/amz-crawler" -s dev # 安装依赖 composer require guzzlehttp/guzzle symfony/dom-crawler symfony/css-selector fakerphp/faker
PHP ≥ 8.0 即可，Guzzle 7.x 自带 PSR-18，后续想接 Hyperf 也很方便。
4. 核心流程：从 ASIN → 结构化数组
ASIN → 下载详情页 → 解析静态字段 → 调 AJAX 价格/库存 → 清洗 → CSV/MySQL/Kafka
5. 代码实战：Guzzle + DOMXPath 极速版
php
< ?php
// src/AmzSpider.php
namespace Demo;
use GuzzleHttpClient;
use SymfonyComponentDomCrawlerCrawler;
class AmzSpider
{
private Client $client;
public function __construct()
{
$this- >client = new Client([
'timeout' => 10,
'headers' => [
'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
'Accept-Language' => 'en-US,en;q=0.9',
'Referer' => 'https://www.amazon.com/',
],
]);
}
public function fetchProduct(string $asin): array
{
$url = "https://www.amazon.com/dp/{$asin}";
$html = $this->client->get($url)->getBody()->getContents();
$crawler = new Crawler($html);
$title = $crawler->filter('#productTitle')->text('');
$priceWhole = $crawler->filter('.a-price .a-price-whole')->text('');
$priceFrac = $crawler->filter('.a-price .a-price-fraction')->text('');
$price = trim($priceWhole . '.' . $priceFrac);
$rating = $crawler->filter('#acrPopover')->attr('title');
$rating = $rating ? explode(' ', $rating)[0] : '';
$reviewText = $crawler->filter('#acrCustomerReviewText')->text('');
$reviewCount = filter_var($reviewText, FILTER_SANITIZE_NUMBER_INT);
$imgJson = $crawler->filter('#imgTagWrapperId img')->attr('data-a-dynamic-image');
$imgMap = json_decode($imgJson, true);
$mainImg = $imgMap ? array_key_first($imgMap) : '';
return [
'asin' => $asin,
'title' => trim($title),
'price' => $price,
'rating' => $rating,
'review_count' => (int) $reviewCount,
'main_img' => $mainImg,
'crawled_at' => date('Y-m-d H:i:s'),
];
}
}
CLI 入口文件：
php
#!/usr/bin/env php < ?php require __DIR__ . '/vendor/autoload.php'; use DemoAmzSpider; $asin = $argv[1] ?? 'B08N5WRWNW'; $spider = new AmzSpider(); $data = $spider- >fetchProduct($asin); echo json_encode($data, JSON_UNESCAPED_UNICODE | JSON_PRETTY_PRINT);
运行：
bash
php bin/amz.php B08N5WRWNW
输出示例：
JSON
{
"asin": "B08N5WRWNW",
"title": "Apple AirPods Pro",
"price": "249.00",
"rating": "4.6",
"review_count": 25430,
"main_img": "https://images-na.ssl-images-amazon.com/images/I/71zny7BTRlL._AC_SL1500_.jpg",
"crawled_at": "2025-06-24 18:42:12"
}
6. 反爬三板斧：UA 池、代理池、随机延时
表格
Guzzle 中间件示例：
php
$stack = GuzzleHttpHandlerStack::create();
$stack->push(GuzzleHttpMiddleware::retry(function ($retries, $request, $response, $exception) {
return $retries < 3 && ($exception || $response- >getStatusCode() === 403);
}, function ($retries) {
return 1000 * (2 ** $retries);
}));
7. Selenium 兜底：滑块验证码与动态渲染
当 Amazon 出现滑块时，可用 php-webdriver 驱动 Chrome：
bash
composer require php-webdriver/webdriver
php
$host = 'http://localhost:4444/wd/hub'; // Selenium Standalone
$driver = RemoteWebDriver::create($host, DesiredCapabilities::chrome());
$driver->get("https://www.amazon.com/dp/$asin");
sleep(5);
$html = $driver->getPageSource();
$driver->quit();
技巧：
加载 stealth.min.js 隐藏 WebDriver 特征；
使用 undetected-chromedriver 可过新版滑块。
8. 提速 10 倍：多进程 + Swoole 协程
bash
composer require swoole/swoole
php
use SwooleRuntime;
use SwooleCoroutine;
use function SwooleCoroutinerun;
Runtime::enableCoroutine();
run(function () {
$asins = ['B08N5WRWNW', 'B08L8DKCS1'];
foreach ($asins as $asin) {
Coroutine::create(function () use ($asin) {
$spider = new AmzSpider();
$data = $spider->fetchProduct($asin);
file_put_contents("data/{$asin}.json", json_encode($data));
});
}
});
实测：4 核 8G 机器，协程版 2k SKU/min，CPU 占用 45%。
9. 数据落地：CSV、MySQL、Kafka 一键切换
① CSV（快速验证）
php
$fp = fopen('amz.csv', 'a');
fputcsv($fp, array_keys($data));
fputcsv($fp, $data);
fclose($fp);
② MySQL（生产）
php
$pdo = new PDO('mysql:host=localhost;dbname=amz', 'root', 'root');
$stmt = $pdo->prepare("REPLACE INTO product (asin,title,price,rating,review_count,main_img,crawled_at) VALUES (?,?,?,?,?,?,?)");
$stmt->execute([$data['asin'], $data['title'], $data['price'], $data['rating'], $data['review_count'], $data['main_img'], $data['crawled_at']]);
③ Kafka（实时流）
php
$producer = new KafkaProducer(function() {
return [KafkaProducerConfig::BOOTSTRAP_SERVERS => 'localhost:9092'];
});
$producer->send([
['topic' => 'amz-product', 'value' => json_encode($data), 'key' => $data['asin']]
]);
10. 合规红线：Amazon 爬虫的法律底线
表格
官方替代方案：Amazon Product Advertising API（PA-API 5.0）
稳定、合规、无封 IP 风险
需 Associate Tag + 授权，每日 1w 额度
结论：能 API 不爬虫，能授权不硬刚。
11. 总结与进阶路线
原型阶段：本文代码直接跑，30 行即可出数
扩展阶段：协程池 + 代理池 + 重试，日采 50w SKU
生产阶段：Hyperf + Kafka + ES 实时搜索
商业闭环：价格告警、选品仪表盘、ERP 自动订价
12. 一键运行 & 源码
bash
git clone https://github.com/yourname/amz-php-crawler.git cd amz-php-crawler composer install php bin/amz.php B08N5WRWNW
输出示例：
18:42:12 INFO ASIN=B08N5WRWNW, title=Apple AirPods Pro, price=$249.00, rating=4.6, reviews=25430
如果本文对你有帮助，记得 点赞 + 收藏 + 在看，我们下期「PHP + Swoole 实时价格流」见！
审核编辑 黄宇
