单IP多HTTPS域名场景下的解决方案__最佳实践_HTTPDNS-阿里云
1.背景说明
在搭建支持HTTPS的前端代理服务器时候,通常会遇到让人头痛的证书问题。根据HTTPS的工作原理,浏览器在访问一个HTTPS站点时,先与服务器建立SSL连接,建立连接的第一步就是请求服务器的证书。而服务器在发送证书的时候,是不知道浏览器访问的是哪个域名的,所以不能根据不同域名发送不同的证书。
SNI(Server Name Indication)是为了解决一个服务器使用多个域名和证书的SSL/TLS扩展。它的工作原理如下:
- 在连接到服务器建立SSL链接之前先发送要访问站点的域名(Hostname)。
- 服务器根据这个域名返回一个合适的证书。
目前,大多数操作系统和浏览器都已经很好地支持SNI扩展,OpenSSL 0.9.8也已经内置这一功能。
上述过程中,当客户端使用HTTPDNS解析域名时,请求URL中的host会被替换成HTTPDNS解析出来的IP,导致服务器获取到的域名为解析后的IP,无法找到匹配的证书,只能返回默认的证书或者不返回,所以会出现SSL/TLS握手不成功的错误。
比如当你需要通过HTTPS访问CDN资源时,CDN的站点往往服务了很多的域名,所以需要通过SNI指定具体的域名证书进行通信。
2 解决方案
针对以上问题,可以采用如下方案解决:hook HTTPS访问前SSL连接过程,根据网络请求头部域中的HOST信息,设置SSL连接PeerHost的值,再根据服务器返回的证书执行验证过程。
下面分别列出Android和iOS平台的示例代码。
2.1 Android示例
在HTTPDNS Android Demo Github中针对HttpsURLConnection接口,提供了在SNI业务场景下使用HTTPDNS的示例代码。
定制SSLSocketFactory,在createSocket时替换为HTTPDNS的IP,并进行SNI/HostNameVerify配置。
class TlsSniSocketFactory extends SSLSocketFactory {
private final String TAG = TlsSniSocketFactory.class.getSimpleName();
HostnameVerifier hostnameVerifier = HttpsURLConnection.getDefaultHostnameVerifier();
private HttpsURLConnection conn;
public TlsSniSocketFactory(HttpsURLConnection conn) {
this.conn = conn;
}
@Override
public Socket createSocket() throws IOException {
return null;
}
@Override
public Socket createSocket(String host, int port) throws IOException, UnknownHostException {
return null;
}
@Override
public Socket createSocket(String host, int port, InetAddress localHost, int localPort) throws IOException, UnknownHostException {
return null;
}
@Override
public Socket createSocket(InetAddress host, int port) throws IOException {
return null;
}
@Override
public Socket createSocket(InetAddress address, int port, InetAddress localAddress, int localPort) throws IOException {
return null;
}
// TLS layer
@Override
public String[] getDefaultCipherSuites() {
return new String[0];
}
@Override
public String[] getSupportedCipherSuites() {
return new String[0];
}
@Override
public Socket createSocket(Socket plainSocket, String host, int port, boolean autoClose) throws IOException {
String peerHost = this.conn.getRequestProperty("Host");
if (peerHost == null)
peerHost = host;
Log.i(TAG, "customized createSocket. host: " + peerHost);
InetAddress address = plainSocket.getInetAddress();
if (autoClose) {
// we don't need the plainSocket
plainSocket.close();
}
// create and connect SSL socket, but don't do hostname/certificate verification yet
SSLCertificateSocketFactory sslSocketFactory = (SSLCertificateSocketFactory) SSLCertificateSocketFactory.getDefault(0);
SSLSocket ssl = (SSLSocket) sslSocketFactory.createSocket(address, port);
// enable TLSv1.1/1.2 if available
ssl.setEnabledProtocols(ssl.getSupportedProtocols());
// set up SNI before the handshake
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
Log.i(TAG, "Setting SNI hostname");
sslSocketFactory.setHostname(ssl, peerHost);
} else {
Log.d(TAG, "No documented SNI support on Android <4.2, trying with reflection");
try {
java.lang.reflect.Method setHostnameMethod = ssl.getClass().getMethod("setHostname", String.class);
setHostnameMethod.invoke(ssl, peerHost);
} catch (Exception e) {
Log.w(TAG, "SNI not useable", e);
}
}
// verify hostname and certificate
SSLSession session = ssl.getSession();
if (!hostnameVerifier.verify(peerHost, session))
throw new SSLPeerUnverifiedException("Cannot verify hostname: " + peerHost);
Log.i(TAG, "Established " + session.getProtocol() + " connection with " + session.getPeerHost() +
" using " + session.getCipherSuite());
return ssl;
}
}
对于需要设置SNI的站点,通常需要重定向请求,示例中也给出了重定向请求的处理方法。
public void recursiveRequest(String path, String reffer) {
URL url = null;
try {
url = new URL(path);
conn = (HttpsURLConnection) url.openConnection();
// 同步接口获取IP
String ip = httpdns.getIpByHostAsync(url.getHost());
if (ip != null) {
// 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
Log.d(TAG, "Get IP: " + ip + " for host: " + url.getHost() + " from HTTPDNS successfully!");
String newUrl = path.replaceFirst(url.getHost(), ip);
conn = (HttpsURLConnection) new URL(newUrl).openConnection();
// 设置HTTP请求头Host域
conn.setRequestProperty("Host", url.getHost());
}
conn.setConnectTimeout(30000);
conn.setReadTimeout(30000);
conn.setInstanceFollowRedirects(false);
TlsSniSocketFactory sslSocketFactory = new TlsSniSocketFactory(conn);
conn.setSSLSocketFactory(sslSocketFactory);
conn.setHostnameVerifier(new HostnameVerifier() {
/*
* 关于这个接口的说明,官方有文档描述:
* This is an extended verification option that implementers can provide.
* It is to be used during a handshake if the URL's hostname does not match the
* peer's identification hostname.
*
* 使用HTTPDNS后URL里设置的hostname不是远程的主机名(如:m.taobao.com),与证书颁发的域不匹配,
* Android HttpsURLConnection提供了回调接口让用户来处理这种定制化场景。
* 在确认HTTPDNS返回的源站IP与Session携带的IP信息一致后,您可以在回调方法中将待验证域名替换为原来的真实域名进行验证。
*
*/
@Override
public boolean verify(String hostname, SSLSession session) {
String host = conn.getRequestProperty("Host");
if (null == host) {
host = conn.getURL().getHost();
}
return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);
}
});
int code = conn.getResponseCode();// Network block
if (needRedirect(code)) {
//临时重定向和永久重定向location的大小写有区分
String location = conn.getHeaderField("Location");
if (location == null) {
location = conn.getHeaderField("location");
}
if (!(location.startsWith("https://") || location
.startsWith("https://"))) {
//某些时候会省略host,只返回后面的path,所以需要补全url
URL originalUrl = new URL(path);
location = originalUrl.getProtocol() + "://"
+ originalUrl.getHost() + location;
}
recursiveRequest(location, path);
} else {
// redirect finish.
DataInputStream dis = new DataInputStream(conn.getInputStream());
int len;
byte[] buff = new byte[4096];
StringBuilder response = new StringBuilder();
while ((len = dis.read(buff)) != -1) {
response.append(new String(buff, 0, len));
}
Log.d(TAG, "Response: " + response.toString());
}
} catch (MalformedURLException e) {
Log.w(TAG, "recursiveRequest MalformedURLException");
} catch (IOException e) {
Log.w(TAG, "recursiveRequest IOException");
} catch (Exception e) {
Log.w(TAG, "unknow exception");
} finally {
if (conn != null) {
conn.disconnect();
}
}
}
private boolean needRedirect(int code) {
return code >= 300 && code < 400;
}
2.2 iOS示例
由于iOS系统并没有提供设置SNI的上层接口(NSURLConnection/NSURLSession),因此在HTTPDNS iOS Demo Github中,我们使用NSURLProtocol拦截网络请求,然后使用CFHTTPMessageRef创建NSInputStream实例进行Socket通信,并设置其kCFStreamSSLPeerName的值。
需要注意的是,使用NSURLProtocol拦截NSURLSession发起的POST请求时,HTTPBody为空。解决方案有两个:
- 使用NSURLConnection发POST请求。
- 先将HTTPBody放入HTTP Header field中,然后在NSURLProtocol中再取出来,Demo中主要演示该方案。
部分代码如下:
在网络请求前注册NSURLProtocol子类,在示例的SNIViewController.m中。
// 注册拦截请求的NSURLProtocol
[NSURLProtocol registerClass:[CFHttpMessageURLProtocol class]];
// 初始化HTTPDNS
HttpDnsService *httpdns = [HttpDnsService sharedInstance];
// 需要设置SNI的URL
NSString *originalUrl = @"your url";
NSURL *url = [NSURL URLWithString:originalUrl];
self.request = [[NSMutableURLRequest alloc] initWithURL:url];
NSString *ip = [httpdns getIpByHostAsync:url.host];
// 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
if (ip) {
NSLog(@"Get IP from HTTPDNS Successfully!");
NSRange hostFirstRange = [originalUrl rangeOfString:url.host];
if (NSNotFound != hostFirstRange.location) {
NSString *newUrl = [originalUrl stringByReplacingCharactersInRange:hostFirstRange withString:ip];
self.request.URL = [NSURL URLWithString:newUrl];
[_request setValue:url.host forHTTPHeaderField:@"host"];
}
}
// NSURLConnection例子
[[NSURLConnection alloc] initWithRequest:_request delegate:self startImmediately:YES];
// NSURLSession例子
NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration defaultSessionConfiguration];
NSArray *protocolArray = @[ [CFHttpMessageURLProtocol class] ];
configuration.protocolClasses = protocolArray;
NSURLSession *session = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:[NSOperationQueue mainQueue]];
NSURLSessionTask *task = [session dataTaskWithRequest:_request];
[task resume];
// 注*:使用NSURLProtocol拦截NSURLSession发起的POST请求时,HTTPBody为空。
// 解决方案有两个:1. 使用NSURLConnection发POST请求。
// 2. 先将HTTPBody放入HTTP Header field中,然后在NSURLProtocol中再取出来。
// 下面主要演示第二种解决方案
// NSString *postStr = [NSString stringWithFormat:@"param1=%@¶m2=%@", @"val1", @"val2"];
// [_request addValue:postStr forHTTPHeaderField:@"originalBody"];
// _request.HTTPMethod = @"POST";
// NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration defaultSessionConfiguration];
// NSArray *protocolArray = @[ [CFHttpMessageURLProtocol class] ];
// configuration.protocolClasses = protocolArray;
// NSURLSession *session = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:[NSOperationQueue mainQueue]];
// NSURLSessionTask *task = [session dataTaskWithRequest:_request];
// [task resume];
在NSURLProtocol子类中拦截网络请求,在示例的CFHttpMessageURLProtocol.m中。
+ (BOOL)canInitWithRequest:(NSURLRequest *)request {
/* 防止无限循环,因为一个请求在被拦截处理过程中,也会发起一个请求,这样又会走到这里,如果不进行处理,就会造成无限循环 */
if ([NSURLProtocol propertyForKey:protocolKey inRequest:request]) {
return NO;
}
NSString *url = request.URL.absoluteString;
// 如果url以https开头,则进行拦截处理,否则不处理
if ([url hasPrefix:@"https"]) {
return YES;
}
return NO;
}
使用CFHTTPMessageRef创建NSInputStream,并设置kCFStreamSSLPeerName,重新发起请求。
/**
* 开始加载,在该方法中,加载一个请求
*/
- (void)startLoading {
NSMutableURLRequest *request = [self.request mutableCopy];
// 表示该请求已经被处理,防止无限循环
[NSURLProtocol setProperty:@(YES) forKey:protocolKey inRequest:request];
curRequest = request;
[self startRequest];
}
/**
* 取消请求
*/
- (void)stopLoading {
if (inputStream.streamStatus == NSStreamStatusOpen) {
[inputStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];
[inputStream setDelegate:nil];
[inputStream close];
}
[self.client URLProtocol:self didFailWithError:[[NSError alloc] initWithDomain:@"stop loading" code:-1 userInfo:nil]];
}
/**
* 使用CFHTTPMessage转发请求
*/
- (void)startRequest {
// 原请求的header信息
NSDictionary *headFields = curRequest.allHTTPHeaderFields;
// 添加http post请求所附带的数据
CFStringRef requestBody = CFSTR("");
CFDataRef bodyData = CFStringCreateExternalRepresentation(kCFAllocatorDefault, requestBody, kCFStringEncodingUTF8, 0);
if (curRequest.HTTPBody) {
bodyData = (__bridge_retained CFDataRef) curRequest.HTTPBody;
} else if (headFields[@"originalBody"]) {
// 使用NSURLSession发POST请求时,将原始HTTPBody从header中取出
bodyData = (__bridge_retained CFDataRef) [headFields[@"originalBody"] dataUsingEncoding:NSUTF8StringEncoding];
}
CFStringRef url = (__bridge CFStringRef) [curRequest.URL absoluteString];
CFURLRef requestURL = CFURLCreateWithString(kCFAllocatorDefault, url, NULL);
// 原请求所使用的方法,GET或POST
CFStringRef requestMethod = (__bridge_retained CFStringRef) curRequest.HTTPMethod;
// 根据请求的url、方法、版本创建CFHTTPMessageRef对象
CFHTTPMessageRef cfrequest = CFHTTPMessageCreateRequest(kCFAllocatorDefault, requestMethod, requestURL, kCFHTTPVersion1_1);
CFHTTPMessageSetBody(cfrequest, bodyData);
// copy原请求的header信息
for (NSString *header in headFields) {
if (![header isEqualToString:@"originalBody"]) {
// 不包含POST请求时存放在header的body信息
CFStringRef requestHeader = (__bridge CFStringRef) header;
CFStringRef requestHeaderValue = (__bridge CFStringRef) [headFields valueForKey:header];
CFHTTPMessageSetHeaderFieldValue(cfrequest, requestHeader, requestHeaderValue);
}
}
// 创建CFHTTPMessage对象的输入流
CFReadStreamRef readStream = CFReadStreamCreateForHTTPRequest(kCFAllocatorDefault, cfrequest);
inputStream = (__bridge_transfer NSInputStream *) readStream;
// 设置SNI host信息,关键步骤
NSString *host = [curRequest.allHTTPHeaderFields objectForKey:@"host"];
if (!host) {
host = curRequest.URL.host;
}
[inputStream setProperty:NSStreamSocketSecurityLevelNegotiatedSSL forKey:NSStreamSocketSecurityLevelKey];
NSDictionary *sslProperties = [[NSDictionary alloc] initWithObjectsAndKeys:
host, (__bridge id) kCFStreamSSLPeerName,
nil];
[inputStream setProperty:sslProperties forKey:(__bridge_transfer NSString *) kCFStreamPropertySSLSettings];
[inputStream setDelegate:self];
if (!curRunLoop)
// 保存当前线程的runloop,这对于重定向的请求很关键
curRunLoop = [NSRunLoop currentRunLoop];
// 将请求放入当前runloop的事件队列
[inputStream scheduleInRunLoop:curRunLoop forMode:NSRunLoopCommonModes];
[inputStream open];
CFRelease(cfrequest);
CFRelease(requestURL);
CFRelease(url);
cfrequest = NULL;
CFRelease(bodyData);
CFRelease(requestBody);
CFRelease(requestMethod);
}
在NSStream的回调函数中,根据不同的eventCode通知原请求。此处eventCode主要有四种:
- NSStreamEventOpenCompleted:连接已打开。
- NSStreamEventHasBytesAvailable:响应头部已下载完整,body中字节可读。
- NSStreamEventErrorOccurred:连接发生错误。
- NSStreamEventEndEncountered:连接结束。
- (void)stream:(NSStream *)aStream handleEvent:(NSStreamEvent)eventCode {
if (eventCode == NSStreamEventHasBytesAvailable) {
CFReadStreamRef readStream = (__bridge_retained CFReadStreamRef) aStream;
CFHTTPMessageRef message = (CFHTTPMessageRef) CFReadStreamCopyProperty(readStream, kCFStreamPropertyHTTPResponseHeader);
if (CFHTTPMessageIsHeaderComplete(message)) {
// 以防response的header信息不完整
UInt8 buffer[16 * 1024];
UInt8 *buf = NULL;
unsigned long length = 0;
NSInputStream *inputstream = (NSInputStream *) aStream;
NSNumber *alreadyAdded = objc_getAssociatedObject(aStream, kAnchorAlreadyAdded);
if (!alreadyAdded || ![alreadyAdded boolValue]) {
objc_setAssociatedObject(aStream, kAnchorAlreadyAdded, [NSNumber numberWithBool:YES], OBJC_ASSOCIATION_COPY);
// 通知client已收到response,只通知一次
NSDictionary *headDict = (__bridge NSDictionary *) (CFHTTPMessageCopyAllHeaderFields(message));
CFStringRef httpVersion = CFHTTPMessageCopyVersion(message);
// 获取响应头部的状态码
CFIndex myErrCode = CFHTTPMessageGetResponseStatusCode(message);
NSHTTPURLResponse *response = [[NSHTTPURLResponse alloc] initWithURL:curRequest.URL statusCode:myErrCode HTTPVersion:(__bridge NSString *) httpVersion headerFields:headDict];
[self.client URLProtocol:self didReceiveResponse:response cacheStoragePolicy:NSURLCacheStorageNotAllowed];
// 验证证书
SecTrustRef trust = (__bridge SecTrustRef) [aStream propertyForKey:(__bridge NSString *) kCFStreamPropertySSLPeerTrust];
SecTrustResultType res = kSecTrustResultInvalid;
NSMutableArray *policies = [NSMutableArray array];
NSString *domain = [[curRequest allHTTPHeaderFields] valueForKey:@"host"];
if (domain) {
[policies addObject:(__bridge_transfer id) SecPolicyCreateSSL(true, (__bridge CFStringRef) domain)];
} else {
[policies addObject:(__bridge_transfer id) SecPolicyCreateBasicX509()];
}
/*
* 绑定校验策略到服务端的证书上
*/
SecTrustSetPolicies(trust, (__bridge CFArrayRef) policies);
if (SecTrustEvaluate(trust, &res) != errSecSuccess) {
[aStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];
[aStream setDelegate:nil];
[aStream close];
[self.client URLProtocol:self didFailWithError:[[NSError alloc] initWithDomain:@"can not evaluate the server trust" code:-1 userInfo:nil]];
}
if (res != kSecTrustResultProceed && res != kSecTrustResultUnspecified) {
/* 证书验证不通过,关闭input stream */
[aStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];
[aStream setDelegate:nil];
[aStream close];
[self.client URLProtocol:self didFailWithError:[[NSError alloc] initWithDomain:@"fail to evaluate the server trust" code:-1 userInfo:nil]];
} else {
// 证书通过,返回数据
if (![inputstream getBuffer:&buf length:&length]) {
NSInteger amount = [inputstream read:buffer maxLength:sizeof(buffer)];
buf = buffer;
length = amount;
}
NSData *data = [[NSData alloc] initWithBytes:buf length:length];
[self.client URLProtocol:self didLoadData:data];
}
} else {
// 证书已验证过,返回数据
if (![inputstream getBuffer:&buf length:&length]) {
NSInteger amount = [inputstream read:buffer maxLength:sizeof(buffer)];
buf = buffer;
length = amount;
}
NSData *data = [[NSData alloc] initWithBytes:buf length:length];
[self.client URLProtocol:self didLoadData:data];
}
}
} else if (eventCode == NSStreamEventErrorOccurred) {
[aStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];
[aStream setDelegate:nil];
[aStream close];
// 通知client发生错误了
[self.client URLProtocol:self didFailWithError:[aStream streamError]];
} else if (eventCode == NSStreamEventEndEncountered) {
[self handleResponse];
}
}
- (void)handleResponse {
// 获取响应头部信息
CFReadStreamRef readStream = (__bridge_retained CFReadStreamRef) inputStream;
CFHTTPMessageRef message = (CFHTTPMessageRef) CFReadStreamCopyProperty(readStream, kCFStreamPropertyHTTPResponseHeader);
if (CFHTTPMessageIsHeaderComplete(message)) {
// 确保response头部信息完整
NSDictionary *headDict = (__bridge NSDictionary *) (CFHTTPMessageCopyAllHeaderFields(message));
// 获取响应头部的状态码
CFIndex myErrCode = CFHTTPMessageGetResponseStatusCode(message);
// 把当前请求关闭
[inputStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];
[inputStream setDelegate:nil];
[inputStream close];
if (myErrCode >= 200 && myErrCode < 300) {
// 返回码为2xx,直接通知client
[self.client URLProtocolDidFinishLoading:self];
} else if (myErrCode >= 300 && myErrCode < 400) {
// 返回码为3xx,需要重定向请求,继续访问重定向页面
NSString *location = headDict[@"Location"];
NSURL *url = [[NSURL alloc] initWithString:location];
curRequest = [[NSMutableURLRequest alloc] initWithURL:url];
/***********重定向通知client处理或内部处理*************/
// client处理
// NSURLResponse* response = [[NSURLResponse alloc] initWithURL:curRequest.URL MIMEType:headDict[@"Content-Type"] expectedContentLength:[headDict[@"Content-Length"] integerValue] textEncodingName:@"UTF8"];
// [self.client URLProtocol:self wasRedirectedToRequest:curRequest redirectResponse:response];
// 内部处理,将url中的host通过HTTPDNS转换为IP,不能在startLoading线程中进行同步网络请求,会被阻塞
NSString *ip = [[HttpDnsService sharedInstance] getIpByHostAsync:url.host];
if (ip) {
NSLog(@"Get IP from HTTPDNS Successfully!");
NSRange hostFirstRange = [location rangeOfString:url.host];
if (NSNotFound != hostFirstRange.location) {
NSString *newUrl = [location stringByReplacingCharactersInRange:hostFirstRange withString:ip];
curRequest = [[NSMutableURLRequest alloc] initWithURL:[NSURL URLWithString:newUrl]];
[curRequest setValue:url.host forHTTPHeaderField:@"host"];
}
}
[self startRequest];
} else {
// 其他情况,直接返回响应信息给client
[self.client URLProtocolDidFinishLoading:self];
}
} else {
// 头部信息不完整,关闭inputstream,通知client
[inputStream removeFromRunLoop:curRunLoop forMode:NSRunLoopCommonModes];
[inputStream setDelegate:nil];
[inputStream close];
[self.client URLProtocolDidFinishLoading:self];
}
}
立即试用HTTPDNS服务,点击此处。
最后更新:2016-11-23 16:04:14
上一篇:
HTTPDNS域名解析场景下如何使用Cookie?__最佳实践_HTTPDNS-阿里云
下一篇:
金融云特性__金融云介绍_金融云-阿里云
常见错误说明__附录_大数据计算服务-阿里云
发送短信接口__API使用手册_短信服务-阿里云
接口文档__Android_安全组件教程_移动安全-阿里云
运营商错误码(联通)__常见问题_短信服务-阿里云
设置短信模板__使用手册_短信服务-阿里云
OSS 权限问题及排查__常见错误及排除_最佳实践_对象存储 OSS-阿里云
消息通知__操作指南_批量计算-阿里云
设备端快速接入(MQTT)__快速开始_阿里云物联网套件-阿里云
查询API调用流量数据__API管理相关接口_API_API 网关-阿里云
使用STS访问__JavaScript-SDK_SDK 参考_对象存储 OSS-阿里云