Home » Review Guidelines

Review Guidelines

How we measure the quality of a Best VPN Service

BestVPNReviews is committed to provide users with an unbiased independent analysis of paid and free VPN services worldwide. We do not accept payments from VPN services nor do we accept subscriptions other than by paying for them ourselves, like any other user.

As an independent review site and per our policy we do not provide VPN services with notice of our tests. This is an unregulated industry and no standards or guidelines have been set so that consumers can determine precisely whether a VPN service is protecting or harming them. Most consumers are not in a position to technically evaluate a VPN service. As the only independent review site not associated with VPN services, we act as a watchdog to provide consumers with information that may contradict the marketing claims of VPN services so that consumers can make an informed decision.

After our review is posted, we provide the reviewed VPN service with an opportunity to respond to the review and to post their comments on the blog immediately after the review.

To better understand how we judge and qualify best VPN services, please read these Guidelines. Each section is given different weight depending on its importance to underlying security and risk to the user. At the end of these Guidelines is a technical guide setting out the methodology for testing.

We do not distinguish the age of VPN services. A new service properly capitalized with a good offering may outlast a service that has been around for some time but has not broken even. At the time of this writing all VPN services are private companies that don’t disclose financial information, therefore the only determination of future viability must be based on the quality of the product offering.

From Order to Installation

We like to see a smooth transition from ordering to installation without any headaches or having to go back to support just to install the application. VPN services that provide OpenVPN standard fare require their users to find the right installation version from the OpenVPN.org site and then to separately download server certificates to the correct config file in the users’ directory. That is a beta product, not a product for which users should pay for.

We prefer to see a service’s own electronic download package that runs quickly with no glitches and without any user involvement. Applications should come with software certificates so that users don’t have to approve the download.

Ease of Use

From ordering to installation to the use of the application: user friendly, convenient and secure determines the points awarded. If the ordering process and installation are a nightmare but the application works well, a lower score can be expected.

Software/GUI and Features

OpenVPN.org provides the open source standard that VPN services are currently using. The standard application is featureless, inconvenient and in some ways unsafe because the loss of underlying connection can terminate the OpenVPN encrypted connection leaving the user vulnerable.

VPN services that do not enhance their application and add important features are downgraded.
Applications that require multiple clicks to start and/or to switch servers, are inconvenient or overly complicated are downgraded.

We like to see applications that are intuitive, slick and easy to use with many features. Features should allow you to:

• Easily switch servers
• Protect against an underlying connection failure
• Clearly see an encrypted secure connection
• Clearly see an unsecure connection
• Automate IP changes
• Determine automatically the fastest (lowest latency) server near you
• Change specific IPs from static to dynamic to fixed IPs
• To use the same Fixed IP during the term of the subscription
• Automatically start the application on startup
• To choose a specific server at automatic startup
• Choose other features that are useful, security enhancing and convenient.

Server Network, Availability, Location & Speed

Best VPN services are only as good as their servers. Great applications but poor servers in the wrong locations do not add up to a good offering. A good balance is what is called for.

Servers should be:

• Fast and are tested for ping times against servers in the same locations.
• Available and not down excessively
• Well load balanced
• Not subject to Spam reports/blacklisted. Services that are blacklisted will not survive long because their hosts (ISPs) will shut them down
• In locations in non-cooperative regions within a short latency from the user’s location. That means if the user is in the USA, some servers should be in Latin America or the Caribbean for users that require confidential exit servers to secure their communications and data. For Europeans, servers located in Russia, Eastern Europe and the Middle East may serve the same purpose. For Asians, servers in non cooperative regions are fairly easy to match to low latency.

What is important is that users need not rely on the privacy protestations of VPN services. Many services claim that because they don’t log or cache, user’s data is safe from regulators or third parties. But one cannot know for sure if the service logs or caches but you can be sure that if legally enforced, a VPN service will log and cache and a government agency may have access to all the data on a specific server (not just of the target but ALL the data if ruled by a Judge). To be completely secure, choose a VPN provider whose corporate headquarters are off shore to begin with. USA VPNs that are also ISPs or are prominently located in the US or Europe are not secure by definition.

If a VPN service’s server IPs are subject to SPAM reports and are blacklisted, we take a very dim view of the service. If the service can’t police its servers and enforce sufficient safeguards to prevent wide scale SPAMMING, then that service will be out of business in no time because its server hosts will have no choice but to ban them.

Presale Support to Product Help & Support

We prefer a VPN service to have a ticket support system, live chat, 7×24 support, a knowledge base and serious FAQS. Generally most VPNs fail somewhat in this category. Even if a VPN service provides good and timely email support, should the service encounter problems along the way, it must have other forms of support backup to perform adequately in times of stress.

If presale support is great but fails upon subscription that suggests that the company is a marketing gimmick and may not be serious about the long term confidence of its users.

We like to see VPN services provide time estimates for responses and meet those estimates. It’s better to state we provide support within 12 hours and respond in 2 than vice versa.

Scope of Protection

Protocols

Encryption protocols vary and some are more susceptible to attack and/or termination by governments. In our view, OpenVPN is best and the higher the encryption strength the better. But that said, 128 Bit is extremely strong and allows the VPN service to provide faster connections. There are comments in the literature that AES may be vulnerable to certain attacks, but none have been measured in the field and supercomputing resources necessary to try such attacks is beyond the scope of nearly all governments and companies and would only be worthwhile against serious terrorist threats.

In our view it is better if a VPN service provides a choice between 128 and 256 bit encryption as the level of threat to encryption systems is generally not known until after they have been broken.

In general shared IPs are best because at the exit server the decrypted data packets are randomly mixed and sent on their way. Dynamic IPs could result in a user’s packets being the sole group of packets last out the door. Fixed IPs are less good but absolutely necessary for connections to PayPal and credit card vendors. Otherwise clients will find their accounts suspended. So a good solution is that the best VPN service provides Shared IPs for all data and communications connections but that the client can switch conveniently to a Fixed IP for specific connections.

As IPv6 gets underway and becomes a default setting, VPN services will have to support this protocol.

Multi hopping, particularly between non cooperative regions is extremely helpful to reduce risk and improve security.

Encryption

If the connection is not encrypted, the security value is nil. Our methodology for testing encrypted connections is provided as an appendix to these Guidelines.

Some VPN services’ encrypted connections fail on certain versions of Windows, specifically Windows 7 and Vista. When they fail, the underlying IP is immediately revealed unbeknown to the user. This is a most grievous fault and heavily weighted to the negative.

Server Security

Another weak link in many VPN services is the security of the servers. Can the servers be breached and the architecture of the network dissected and read? If that is the case, hackers can see the originating IPs as well as any other personal data logged or cached by the service.

• Are the servers PCI complaint and provide data about the safety of the credit card payments
• Have PCI compliance reported security holes, warnings and notes and been fixed?
• Observe credit card gateway security policies
• Are there high Level network vulnerabilities like DDOS or router or firewall vulnerabilities?
• Have the server software and its components been properly tested? What patches are applied?
• Corporate web site audit to detect XSS, SQL injections, CGI and other bugs.
• User side security for login rules
• Does server admin comply with Information Systems Audit and Control Foundations (ISACF)

These and other tests are performed by BestVPNReviews to determine server security of the VPN Services and points are deducted for deficiencies. We ask permission from each VPN service to test the servers and network before we perform any test or survey. If a VPN service does not permit us to evaluate server security, we deduct 50% from the server security points available.

Privacy

Like encryption, server security and location, privacy goes to the heart of the matter with respect to VPN services. The problem, of course, is how to test for a VPN services’ privacy policy compliance. As there is no objective test, it boils down to trust and verify. If we find any behavior that suggests lack of trust, we no longer take comfort in the written word of the privacy policy. So if the ordering process suggests trickiness or obfuscation, we tend to believe less in the VPN service as a whole.

If a VPN service pays for its service through advertising and claims that the advertiser is not able to obtain information about the user, we balk at the bold faced lie. Advertisers place code (both session and persistent cookies) on the browser which can be used to detect the underlying IP as well as browser history. So if you expect any privacy from a free VPN provider that uses advertising as a business model, don’t expect any privacy whatsoever. Your personal information or data will be resident on thousands of advertising servers from the get go. So don’t be surprised. VPN security and privacy do not swim in the same sea as advertisers.

With respect to the privacy policy itself, we like to see clear no-nonsense language which clarifies completely the intent of the VPN provider. If the policy says we don’t log but will go after you if you commit a crime that gives us a problem. Either you log or you don’t log. To go crime busting you need to log and that’s a contradiction of terms. So it’s better to state we log certain data to protect ourselves from users committing crimes and that data is ‘X’. It’s better for the user to know the extent of his risk then to be bamboozled and put to sleep by a misleading privacy policy.

As we wrote above but repeat here because of its importance: users need not rely on the privacy protestations of VPN services. Many services claim that because they don’t log or cache that user’s data is safe from regulators or third parties. But one cannot know if the service logs or caches but you can be sure that if legally enforced, a VPN service will log and cache and a government agency may have access to all the data on a specific server (not just of the target but ALL the data of all the users on that server if ruled by a Judge). No VPN service will conspire against or obstruct a criminal investigation that results in a sealed court order that instructs management of that VPN service to log and cache data at specified servers and that includes off shore servers that are under the control of management. Therefore Privacy Policies carry little if any weight. Hushmail in Canada is a case in point (the encrypted email service that terms itself the leader in secure email). When push came to shove, to save their business, Hushmail logged and cached with the best of them to provide evidence against certain clients over a six month period (criminal activities involving the sale of steroids – imagine if it were a serious offense!). If the server were located off shore, there is a better chance that Hushmail might have been less cooperative. But I wouldn’t count on it.

To be completely secure, choose a VPN provider whose corporate headquarters are off shore to begin with. Many jurisdictions are impervious to US and European court orders. USA VPNs that are also ISPs or are prominently located in the US are not secure by definition. European services in major centers are no better. Most of us feel more comfortable when the services supplied to us come from our country vendors. But this is one case in which nationalism stings. Buying VPN services as an American from a USA based VPN service or doing the same as a European is equivalent to purchasing services from your Internet Service Provider irrespective of what is stated in any privacy policy. If you want privacy, purchase your VPN service off shore.

Furthermore, the more secure the location of the VPN service’s headquarters, the less one has to rely on the privacy policy. And since the privacy policy is based on trust and the blind belief that a VPN service will stand up to a federal prosecutor, the best route is to make sure the VPN service is outside of your jurisdiction and has at least some of its servers off shore.

Governments are beginning to legislate and enforce strict data retention laws throughout the world, including the USA, UK, the European Union and parts of Asia. VPN services will market their product and make bogus claims in the face of this obvious onslaught on privacy. Just read about the current ban being considered against Blackberry in the Middle East. You may think the Middle East is not home, but most governments are sympathetic to the notion of ‘no absolute privacy’ on the grounds of 9/11 and terrorism.

More Information: APPENDIX – VPN Security Testing Methodology