Chinaunix
标题:
我用socket写抓取网页,为什么每个recv都是0字节【已解决】
[打印本页]
作者:
msheng_yeb
时间:
2013-01-10 20:23
标题:
我用socket写抓取网页,为什么每个recv都是0字节【已解决】
本帖最后由 crazyhadoop 于 2013-01-11 08:27 编辑
代码如下
int main(int argc, char *argv[])
{
struct hostent *h;
char *ipAddr = NULL;
char *domain = "www.baidu.com";
if ((h=gethostbyname(domain)) == NULL)
{
herror("gethostbyname");
exit(1);
}
printf("Host name : %s\n", h->h_name);
ipAddr = inet_ntoa(*((struct in_addr*)h->h_addr));
printf("IP Address : %s\n", ipAddr);
int client_fd;
client_fd = socket(AF_INET, SOCK_STREAM, 0);
if(client_fd==-1)
{
perror("socket failed\n");
exit(1);
}
struct sockaddr_in server_addr;
memset(&server_addr, 0, sizeof(server_addr));
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(80);
server_addr.sin_addr = *((struct in_addr*)h->h_addr);
int connectState;
connectState = connect(client_fd, (struct sockaddr*)&server_addr, sizeof(server_addr));
if(connectState==-1)
{
perror("conncet failed\n");
}
char msg[1024];
memset(msg, 0, sizeof(msg));
sprintf(msg,"GET http://www.baidu.com/ HTTP/1.1\r\nConnection: Close\r\n\r\n");
printf("%s\n",msg);
int sendState = 0;
sendState = send(client_fd, msg, strlen(msg), 0);
if(sendState==-1)
{
perror("send Error\n");
exit(1);
}
int receiveLen = 0;
char receiveBuffer[1024];
while(true)
{
receiveLen = recv(client_fd, receiveBuffer,sizeof(receiveBuffer), 0);
printf("receive len = %d\n", receiveLen);
if(receiveLen<0)
break;
}
close(client_fd);
return 0;
}
复制代码
从printf("receive len = %d\n", receiveLen);看出,打印出无限个receiveLen等于0的值
请问是我send的数据有问题吗?
作者:
msheng_yeb
时间:
2013-01-10 22:27
自问自答吧,应该是send的数据有问题
char msg[1024];
memset(msg, 0, sizeof(msg));
sprintf(msg,"GET ");
strcat(msg, url);
strcat(msg, " HTTP/1.1");
strcat(msg, "\r\nHost: ");
strcat(msg, domain);
strcat(msg, "\r\nUser-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20100101 Firefox/15.0.1");
strcat(msg, "\r\nAccept: */*");
strcat(msg, "\r\nConnection: close\r\n\r\n");
复制代码
换成上面的代码就OK抓到网页了
url=http://www.baidu.com/
domain=www.baidu.com
作者:
thelordsaves
时间:
2013-01-15 09:52
不错!
欢迎光临 Chinaunix (http://bbs.chinaunix.net/)
Powered by Discuz! X3.2