单IP多HTTPS域名场景下的解决方案__最佳实践_HTTPDNS-阿里云
1.背景说明
在搭建支持HTTPS的前端代理服务器时候,通常会遇到让人头痛的证书问题。根据HTTPS的工作原理,浏览器在访问一个HTTPS站点时,先与服务器建立SSL连接,建立连接的第一步就是请求服务器的证书。而服务器在发送证书的时候,是不知道浏览器访问的是哪个域名的,所以不能根据不同域名发送不同的证书。
SNI(Server Name Indication)是为了解决一个服务器使用多个域名和证书的SSL/TLS扩展。它的工作原理如下:
- 在连接到服务器建立SSL链接之前先发送要访问站点的域名(Hostname)。
- 服务器根据这个域名返回一个合适的证书。
目前,大多数操作系统和浏览器都已经很好地支持SNI扩展,OpenSSL 0.9.8也已经内置这一功能。
上述过程中,当客户端使用HTTPDNS解析域名时,请求URL中的host会被替换成HTTPDNS解析出来的IP,导致服务器获取到的域名为解析后的IP,无法找到匹配的证书,只能返回默认的证书或者不返回,所以会出现SSL/TLS握手不成功的错误。
比如当你需要通过HTTPS访问CDN资源时,CDN的站点往往服务了很多的域名,所以需要通过SNI指定具体的域名证书进行通信。
2 解决方案
针对以上问题,可以采用如下方案解决:hook HTTPS访问前SSL连接过程,根据网络请求头部域中的HOST信息,设置SSL连接PeerHost的值,再根据服务器返回的证书执行验证过程。
下面分别列出Android和iOS平台的示例代码。
2.1 Android示例
在HTTPDNS Android Demo Github中针对HttpsURLConnection接口,提供了在SNI业务场景下使用HTTPDNS的示例代码。
定制SSLSocketFactory,在createSocket时替换为HTTPDNS的IP,并进行SNI/HostNameVerify配置。
class TlsSniSocketFactory extends SSLSocketFactory {private final String TAG = TlsSniSocketFactory.class.getSimpleName();HostnameVerifier hostnameVerifier = HttpsURLConnection.getDefaultHostnameVerifier();private HttpsURLConnection conn;public TlsSniSocketFactory(HttpsURLConnection conn) {this.conn = conn;}@Overridepublic Socket createSocket() throws IOException {return null;}@Overridepublic Socket createSocket(String host, int port) throws IOException, UnknownHostException {return null;}@Overridepublic Socket createSocket(String host, int port, InetAddress localHost, int localPort) throws IOException, UnknownHostException {return null;}@Overridepublic Socket createSocket(InetAddress host, int port) throws IOException {return null;}@Overridepublic Socket createSocket(InetAddress address, int port, InetAddress localAddress, int localPort) throws IOException {return null;}// TLS layer@Overridepublic String[] getDefaultCipherSuites() {return new String[0];}@Overridepublic String[] getSupportedCipherSuites() {return new String[0];}@Overridepublic Socket createSocket(Socket plainSocket, String host, int port, boolean autoClose) throws IOException {String peerHost = this.conn.getRequestProperty("Host");if (peerHost == null)peerHost = host;Log.i(TAG, "customized createSocket. host: " + peerHost);InetAddress address = plainSocket.getInetAddress();if (autoClose) {// we don't need the plainSocketplainSocket.close();}// create and connect SSL socket, but don't do hostname/certificate verification yetSSLCertificateSocketFactory sslSocketFactory = (SSLCertificateSocketFactory) SSLCertificateSocketFactory.getDefault(0);SSLSocket ssl = (SSLSocket) sslSocketFactory.createSocket(address, port);// enable TLSv1.1/1.2 if availablessl.setEnabledProtocols(ssl.getSupportedProtocols());// set up SNI before the handshakeif (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {Log.i(TAG, "Setting SNI hostname");sslSocketFactory.setHostname(ssl, peerHost);} else {Log.d(TAG, "No documented SNI support on Android <4.2, trying with reflection");try {java.lang.reflect.Method setHostnameMethod = ssl.getClass().getMethod("setHostname", String.class);setHostnameMethod.invoke(ssl, peerHost);} catch (Exception e) {Log.w(TAG, "SNI not useable", e);}}// verify hostname and certificateSSLSession session = ssl.getSession();if (!hostnameVerifier.verify(peerHost, session))throw new SSLPeerUnverifiedException("Cannot verify hostname: " + peerHost);Log.i(TAG, "Established " + session.getProtocol() + " connection with " + session.getPeerHost() +" using " + session.getCipherSuite());return ssl;}}
对于需要设置SNI的站点,通常需要重定向请求,示例中也给出了重定向请求的处理方法。
public void recursiveRequest(String path, String reffer) {URL url = null;try {url = new URL(path);conn = (HttpsURLConnection) url.openConnection();// 同步接口获取IPString ip = httpdns.getIpByHostAsync(url.getHost());if (ip != null) {// 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置Log.d(TAG, "Get IP: " + ip + " for host: " + url.getHost() + " from HTTPDNS successfully!");String newUrl = path.replaceFirst(url.getHost(), ip);conn = (HttpsURLConnection) new URL(newUrl).openConnection();// 设置HTTP请求头Host域conn.setRequestProperty("Host", url.getHost());}conn.setConnectTimeout(30000);conn.setReadTimeout(30000);conn.setInstanceFollowRedirects(false);TlsSniSocketFactory sslSocketFactory = new TlsSniSocketFactory(conn);conn.setSSLSocketFactory(sslSocketFactory);conn.setHostnameVerifier(new HostnameVerifier() {/** 关于这个接口的说明,官方有文档描述:* This is an extended verification option that implementers can provide.* It is to be used during a handshake if the URL's hostname does not match the* peer's identification hostname.** 使用HTTPDNS后URL里设置的hostname不是远程的主机名(如:m.taobao.com),与证书颁发的域不匹配,* Android HttpsURLConnection提供了回调接口让用户来处理这种定制化场景。* 在确认HTTPDNS返回的源站IP与Session携带的IP信息一致后,您可以在回调方法中将待验证域名替换为原来的真实域名进行验证。**/@Overridepublic boolean verify(String hostname, SSLSession session) {String host = conn.getRequestProperty("Host");if (null == host) {host = conn.getURL().getHost();}return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);}});int code = conn.getResponseCode();// Network blockif (needRedirect(code)) {//临时重定向和永久重定向location的大小写有区分String location = conn.getHeaderField("Location");if (location == null) {location = conn.getHeaderField("location");}if (!(location.startsWith("https://") || location.startsWith("https://"))) {//某些时候会省略host,只返回后面的path,所以需要补全urlURL originalUrl = new URL(path);location = originalUrl.getProtocol() + "://"+ originalUrl.getHost() + location;}recursiveRequest(location, path);} else {// redirect finish.DataInputStream dis = new DataInputStream(conn.getInputStream());int len;byte[] buff = new byte[4096];StringBuilder response = new StringBuilder();while ((len = dis.read(buff)) != -1) {response.append(new String(buff, 0, len));}Log.d(TAG, "Response: " + response.toString());}} catch (MalformedURLException e) {Log.w(TAG, "recursiveRequest MalformedURLException");} catch (IOException e) {Log.w(TAG, "recursiveRequest IOException");} catch (Exception e) {Log.w(TAG, "unknow exception");} finally {if (conn != null) {conn.disconnect();}}}private boolean needRedirect(int code) {return code >= 300 && code < 400;}
2.2 iOS示例
由于iOS系统并没有提供设置SNI的上层接口(NSURLConnection/NSURLSession),因此在HTTPDNS iOS Demo Github中,我们使用NSURLProtocol拦截网络请求,然后使用CFHTTPMessageRef创建NSInputStream实例进行Socket通信,并设置其kCFStreamSSLPeerName的值。
需要注意的是,使用NSURLProtocol拦截NSURLSession发起的POST请求时,HTTPBody为空。解决方案有两个:
- 使用NSURLConnection发POST请求。
- 先将HTTPBody放入HTTP Header field中,然后在NSURLProtocol中再取出来,Demo中主要演示该方案。
部分代码如下:
在网络请求前注册NSURLProtocol子类,在示例的SNIViewController.m中。
// 注册拦截请求的NSURLProtocol[NSURLProtocol registerClass:[CFHttpMessageURLProtocol class]];// 初始化HTTPDNSHttpDnsService *httpdns = [HttpDnsService sharedInstance];// 需要设置SNI的URLNSString *originalUrl = @"your url";NSURL *url = [NSURL URLWithString:originalUrl];self.request = [[NSMutableURLRequest alloc] initWithURL:url];NSString *ip = [httpdns getIpByHostAsync:url.host];// 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置if (ip) {NSLog(@"Get IP from HTTPDNS Successfully!");NSRange hostFirstRange = [originalUrl rangeOfString:url.host];if (NSNotFound != hostFirstRange.location) {NSString *newUrl = [originalUrl stringByReplacingCharactersInRange:hostFirstRange withString:ip];self.request.URL = [NSURL URLWithString:newUrl];[_request setValue:url.host forHTTPHeaderField:@"host"];}}// NSURLConnection例子[[NSURLConnection alloc] initWithRequest:_request delegate:self startImmediately:YES];// NSURLSession例子NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration defaultSessionConfiguration];NSArray *protocolArray = @[ [CFHttpMessageURLProtocol class] ];configuration.protocolClasses = protocolArray;NSURLSession *session = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:[NSOperationQueue mainQueue]];NSURLSessionTask *task = [session dataTaskWithRequest:_request];[task resume];// 注*:使用NSURLProtocol拦截NSURLSession发起的POST请求时,HTTPBody为空。// 解决方案有两个:1. 使用NSURLConnection发POST请求。// 2. 先将HTTPBody放入HTTP Header field中,然后在NSURLProtocol中再取出来。// 下面主要演示第二种解决方案// NSString *postStr = [NSString stringWithFormat:@"param1=%@¶m2=%@", @"val1", @"val2"];// [_request addValue:postStr forHTTPHeaderField:@"originalBody"];// _request.HTTPMethod = @"POST";// NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration defaultSessionConfiguration];// NSArray *protocolArray = @[ [CFHttpMessageURLProtocol class] ];// configuration.protocolClasses = protocolArray;// NSURLSession *session = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:[NSOperationQueue mainQueue]];// NSURLSessionTask *task = [session dataTaskWithRequest:_request];// [task resume];
在NSURLProtocol子类中拦截网络请求,在示例的CFHttpMessageURLProtocol.m中。
+ (BOOL)canInitWithRequest:(NSURLRequest *)request {/* 防止无限循环,因为一个请求在被拦截处理过程中,也会发起一个请求,这样又会走到这里,如果不进行处理,就会造成无限循环 */if ([NSURLProtocol propertyForKey:protocolKey inRequest:request]) {return NO;}NSString *url = request.URL.absoluteString;// 如果url以https开头,则进行拦截处理,否则不处理if ([url hasPrefix:@"https"]) {return YES;}return NO;}
使用CFHTTPMessageRef创建NSInputStream,并设置kCFStreamSSLPeerName,重新发起请求。
/*** 开始加载,在该方法中,加载一个请求*/- (void)startLoading {NSMutableURLRequest *request = [self.request mutableCopy];// 表示该请求已经被处理,防止无限循环[NSURLProtocol setProperty:@(YES) forKey:protocolKey inRequest:request];curRequest = request;[self startRequest];}/*** 取消请求*/- (void)stopLoading {if (inputStream.streamStatus == NSStreamStatusOpen) {[inputStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];[inputStream setDelegate:nil];[inputStream close];}[self.client URLProtocol:self didFailWithError:[[NSError alloc] initWithDomain:@"stop loading" code:-1 userInfo:nil]];}/*** 使用CFHTTPMessage转发请求*/- (void)startRequest {// 原请求的header信息NSDictionary *headFields = curRequest.allHTTPHeaderFields;// 添加http post请求所附带的数据CFStringRef requestBody = CFSTR("");CFDataRef bodyData = CFStringCreateExternalRepresentation(kCFAllocatorDefault, requestBody, kCFStringEncodingUTF8, 0);if (curRequest.HTTPBody) {bodyData = (__bridge_retained CFDataRef) curRequest.HTTPBody;} else if (headFields[@"originalBody"]) {// 使用NSURLSession发POST请求时,将原始HTTPBody从header中取出bodyData = (__bridge_retained CFDataRef) [headFields[@"originalBody"] dataUsingEncoding:NSUTF8StringEncoding];}CFStringRef url = (__bridge CFStringRef) [curRequest.URL absoluteString];CFURLRef requestURL = CFURLCreateWithString(kCFAllocatorDefault, url, NULL);// 原请求所使用的方法,GET或POSTCFStringRef requestMethod = (__bridge_retained CFStringRef) curRequest.HTTPMethod;// 根据请求的url、方法、版本创建CFHTTPMessageRef对象CFHTTPMessageRef cfrequest = CFHTTPMessageCreateRequest(kCFAllocatorDefault, requestMethod, requestURL, kCFHTTPVersion1_1);CFHTTPMessageSetBody(cfrequest, bodyData);// copy原请求的header信息for (NSString *header in headFields) {if (![header isEqualToString:@"originalBody"]) {// 不包含POST请求时存放在header的body信息CFStringRef requestHeader = (__bridge CFStringRef) header;CFStringRef requestHeaderValue = (__bridge CFStringRef) [headFields valueForKey:header];CFHTTPMessageSetHeaderFieldValue(cfrequest, requestHeader, requestHeaderValue);}}// 创建CFHTTPMessage对象的输入流CFReadStreamRef readStream = CFReadStreamCreateForHTTPRequest(kCFAllocatorDefault, cfrequest);inputStream = (__bridge_transfer NSInputStream *) readStream;// 设置SNI host信息,关键步骤NSString *host = [curRequest.allHTTPHeaderFields objectForKey:@"host"];if (!host) {host = curRequest.URL.host;}[inputStream setProperty:NSStreamSocketSecurityLevelNegotiatedSSL forKey:NSStreamSocketSecurityLevelKey];NSDictionary *sslProperties = [[NSDictionary alloc] initWithObjectsAndKeys:host, (__bridge id) kCFStreamSSLPeerName,nil];[inputStream setProperty:sslProperties forKey:(__bridge_transfer NSString *) kCFStreamPropertySSLSettings];[inputStream setDelegate:self];if (!curRunLoop)// 保存当前线程的runloop,这对于重定向的请求很关键curRunLoop = [NSRunLoop currentRunLoop];// 将请求放入当前runloop的事件队列[inputStream scheduleInRunLoop:curRunLoop forMode:NSRunLoopCommonModes];[inputStream open];CFRelease(cfrequest);CFRelease(requestURL);CFRelease(url);cfrequest = NULL;CFRelease(bodyData);CFRelease(requestBody);CFRelease(requestMethod);}
在NSStream的回调函数中,根据不同的eventCode通知原请求。此处eventCode主要有四种:
- NSStreamEventOpenCompleted:连接已打开。
- NSStreamEventHasBytesAvailable:响应头部已下载完整,body中字节可读。
- NSStreamEventErrorOccurred:连接发生错误。
- NSStreamEventEndEncountered:连接结束。
- (void)stream:(NSStream *)aStream handleEvent:(NSStreamEvent)eventCode {if (eventCode == NSStreamEventHasBytesAvailable) {CFReadStreamRef readStream = (__bridge_retained CFReadStreamRef) aStream;CFHTTPMessageRef message = (CFHTTPMessageRef) CFReadStreamCopyProperty(readStream, kCFStreamPropertyHTTPResponseHeader);if (CFHTTPMessageIsHeaderComplete(message)) {// 以防response的header信息不完整UInt8 buffer[16 * 1024];UInt8 *buf = NULL;unsigned long length = 0;NSInputStream *inputstream = (NSInputStream *) aStream;NSNumber *alreadyAdded = objc_getAssociatedObject(aStream, kAnchorAlreadyAdded);if (!alreadyAdded || ![alreadyAdded boolValue]) {objc_setAssociatedObject(aStream, kAnchorAlreadyAdded, [NSNumber numberWithBool:YES], OBJC_ASSOCIATION_COPY);// 通知client已收到response,只通知一次NSDictionary *headDict = (__bridge NSDictionary *) (CFHTTPMessageCopyAllHeaderFields(message));CFStringRef httpVersion = CFHTTPMessageCopyVersion(message);// 获取响应头部的状态码CFIndex myErrCode = CFHTTPMessageGetResponseStatusCode(message);NSHTTPURLResponse *response = [[NSHTTPURLResponse alloc] initWithURL:curRequest.URL statusCode:myErrCode HTTPVersion:(__bridge NSString *) httpVersion headerFields:headDict];[self.client URLProtocol:self didReceiveResponse:response cacheStoragePolicy:NSURLCacheStorageNotAllowed];// 验证证书SecTrustRef trust = (__bridge SecTrustRef) [aStream propertyForKey:(__bridge NSString *) kCFStreamPropertySSLPeerTrust];SecTrustResultType res = kSecTrustResultInvalid;NSMutableArray *policies = [NSMutableArray array];NSString *domain = [[curRequest allHTTPHeaderFields] valueForKey:@"host"];if (domain) {[policies addObject:(__bridge_transfer id) SecPolicyCreateSSL(true, (__bridge CFStringRef) domain)];} else {[policies addObject:(__bridge_transfer id) SecPolicyCreateBasicX509()];}/** 绑定校验策略到服务端的证书上*/SecTrustSetPolicies(trust, (__bridge CFArrayRef) policies);if (SecTrustEvaluate(trust, &res) != errSecSuccess) {[aStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];[aStream setDelegate:nil];[aStream close];[self.client URLProtocol:self didFailWithError:[[NSError alloc] initWithDomain:@"can not evaluate the server trust" code:-1 userInfo:nil]];}if (res != kSecTrustResultProceed && res != kSecTrustResultUnspecified) {/* 证书验证不通过,关闭input stream */[aStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];[aStream setDelegate:nil];[aStream close];[self.client URLProtocol:self didFailWithError:[[NSError alloc] initWithDomain:@"fail to evaluate the server trust" code:-1 userInfo:nil]];} else {// 证书通过,返回数据if (![inputstream getBuffer:&buf length:&length]) {NSInteger amount = [inputstream read:buffer maxLength:sizeof(buffer)];buf = buffer;length = amount;}NSData *data = [[NSData alloc] initWithBytes:buf length:length];[self.client URLProtocol:self didLoadData:data];}} else {// 证书已验证过,返回数据if (![inputstream getBuffer:&buf length:&length]) {NSInteger amount = [inputstream read:buffer maxLength:sizeof(buffer)];buf = buffer;length = amount;}NSData *data = [[NSData alloc] initWithBytes:buf length:length];[self.client URLProtocol:self didLoadData:data];}}} else if (eventCode == NSStreamEventErrorOccurred) {[aStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];[aStream setDelegate:nil];[aStream close];// 通知client发生错误了[self.client URLProtocol:self didFailWithError:[aStream streamError]];} else if (eventCode == NSStreamEventEndEncountered) {[self handleResponse];}}- (void)handleResponse {// 获取响应头部信息CFReadStreamRef readStream = (__bridge_retained CFReadStreamRef) inputStream;CFHTTPMessageRef message = (CFHTTPMessageRef) CFReadStreamCopyProperty(readStream, kCFStreamPropertyHTTPResponseHeader);if (CFHTTPMessageIsHeaderComplete(message)) {// 确保response头部信息完整NSDictionary *headDict = (__bridge NSDictionary *) (CFHTTPMessageCopyAllHeaderFields(message));// 获取响应头部的状态码CFIndex myErrCode = CFHTTPMessageGetResponseStatusCode(message);// 把当前请求关闭[inputStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];[inputStream setDelegate:nil];[inputStream close];if (myErrCode >= 200 && myErrCode < 300) {// 返回码为2xx,直接通知client[self.client URLProtocolDidFinishLoading:self];} else if (myErrCode >= 300 && myErrCode < 400) {// 返回码为3xx,需要重定向请求,继续访问重定向页面NSString *location = headDict[@"Location"];NSURL *url = [[NSURL alloc] initWithString:location];curRequest = [[NSMutableURLRequest alloc] initWithURL:url];/***********重定向通知client处理或内部处理*************/// client处理// NSURLResponse* response = [[NSURLResponse alloc] initWithURL:curRequest.URL MIMEType:headDict[@"Content-Type"] expectedContentLength:[headDict[@"Content-Length"] integerValue] textEncodingName:@"UTF8"];// [self.client URLProtocol:self wasRedirectedToRequest:curRequest redirectResponse:response];// 内部处理,将url中的host通过HTTPDNS转换为IP,不能在startLoading线程中进行同步网络请求,会被阻塞NSString *ip = [[HttpDnsService sharedInstance] getIpByHostAsync:url.host];if (ip) {NSLog(@"Get IP from HTTPDNS Successfully!");NSRange hostFirstRange = [location rangeOfString:url.host];if (NSNotFound != hostFirstRange.location) {NSString *newUrl = [location stringByReplacingCharactersInRange:hostFirstRange withString:ip];curRequest = [[NSMutableURLRequest alloc] initWithURL:[NSURL URLWithString:newUrl]];[curRequest setValue:url.host forHTTPHeaderField:@"host"];}}[self startRequest];} else {// 其他情况,直接返回响应信息给client[self.client URLProtocolDidFinishLoading:self];}} else {// 头部信息不完整,关闭inputstream,通知client[inputStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];[inputStream setDelegate:nil];[inputStream close];[self.client URLProtocolDidFinishLoading:self];}}
立即试用HTTPDNS服务,点击此处。
最后更新:2016-11-23 16:04:14
上一篇:
HTTPDNS域名解析场景下如何使用Cookie?__最佳实践_HTTPDNS-阿里云
下一篇:
金融云特性__金融云介绍_金融云-阿里云
常见错误说明__附录_大数据计算服务-阿里云
发送短信接口__API使用手册_短信服务-阿里云
接口文档__Android_安全组件教程_移动安全-阿里云
运营商错误码(联通)__常见问题_短信服务-阿里云
设置短信模板__使用手册_短信服务-阿里云
OSS 权限问题及排查__常见错误及排除_最佳实践_对象存储 OSS-阿里云
消息通知__操作指南_批量计算-阿里云
设备端快速接入(MQTT)__快速开始_阿里云物联网套件-阿里云
查询API调用流量数据__API管理相关接口_API_API 网关-阿里云
使用STS访问__JavaScript-SDK_SDK 参考_对象存储 OSS-阿里云