WhiteWinterWolf Practical IT security, *nix systems & networking

Another company got caught his hand in the cookie jar, and this time we are not talking of the firmware of some cheap home router:

CVE-2017-10151, CVSS 3.0 Base Score 10.0:

Vulnerability in the Oracle Identity Manager component of Oracle Fusion Middleware (subcomponent: Default Account). Supported versions that are affected are 11.1.1.7, 11.1.2.3 and 12.2.1.3.

Easily exploitable vulnerability allows unauthenticated attacker with network access via HTTP to compromise Oracle Identity Manager.

While the vulnerability is in Oracle Identity Manager, attacks may significantly impact additional products.

Successful attacks of this vulnerability can result in takeover of Oracle Identity Manager.

The issue is pretty simple: Oracle added a default account with administrative privileges and hardcoded credentials to their product to alleviate development work. This what is commonly called a backdoor.

While there is obviously no statistics available about such practices, it still seem to happen regularly. On top of my head I can remember of the infamous Alpha Networks / D-Link backdoor which was heavily documented and is studied here: authentication is simply bypassed when the user-agent string equals to “xmlset_roodkcableoj28840ybtide” (read backward, this gives “Edit by 04882 Joel backdoor”).

But the odd thing here in Oracle’s case is that the “feature” was described in the publicly available product documentation:

Account: OIMINTERNAL

Description:

This account is set to a ‘run as’ user for Message Driven Beans (MDBs) executing JMS messages. This account is created during installation and is used internally by Oracle Identity Manager.

The password of this account is set to a single space character in Oracle Identity Manager database to prevent user login through Oracle Identity Manager Design console or Oracle Identity Manager System Administration Console.

Do not change the user name or password of this account.

In general such information is limited to internal, usually informal documentation, the vendor’s bet being that it won’t be publicly discovered until the product gets end-of-life. And indeed, with large and complex software (unlike consumer devices firmwares, but who cares about consumers security? dark irony intended…), overworked teams on customers side and security analysts with too short time frames to do any thorough analysis, most chances are that such flaw will effectively remain unnoticed until vendor’s liability ends.

But why is it so? Is this an evil plan from the NSA to take over the world, as I unsurprisingly read each time such kind of issue pops up? No, this is just the natural result of our current market and economical system.

Making good software is just deemed not profitable.

I regularly encounter people thinking that larger companies, having more money and more people available, should therefore produce better quality software. This is summarized by the classical project management triangle:

It indeed induces that disposing of more money and developer time should logically push software quality onward. The same people are usually more inclined to trust software made by such large companies than FOSS developed by individuals around the world precisely because of this belief.

But this belief is wrong.

A competitive market

Software development is a very competitive market. Large companies are not above such competition, they are at its core:

• Worlwide analysis of business and technological trends such as the Gartner’s Magic Quadrant compare the growing markets and the position of the key actors on each one.

  The goal for each company is to closely follow the market evolution, as determined by such studies¹, and raise themselves to the “Leader” rate and above the other vendors for each targeted markets.

  This forces these companies to be very reactive. In order to remain competitive, they have to be the first offering the most feature-complete solution answering needs which may not even have existed the day before, all this backed by strongest marketing and technical forces.

• Large companies have publicly shared trades whose value and, therefore, investors benefits are directly related to their published quarters results.

  Competitive companies must show the best ROI, which usually goes by reducing the costs to the minimum while adopting the most aggressive marketing techniques to increase sells o the maximum.

  There is no place in this world for temporary reduced profits for the sake of some long-term project. This would be perceived as a weakness. Profit must show a consistent steady increase to keep investors’ trust.

So, for these reasons, software from large editors usually means short development time, low money and high marketing.

But let’s see what this concretely means in terms of software development.

Cost-reduction methods

About the tools

What a lot of people not from the field ignore is that software development is a long and complex process. You have several stages of high and low level design, the actual coding, the debugging which includes unit and integration testing, code refactoring, documentation, etc.

• Coding, the action of “writing” the software, actually plays a very minor role in the whole development process. Adding or changing a feature in a software is therefore far more than simply writing or modifying some code, and the bigger is the software, the larger are the chances that even the smallest change will have side-effects on other features or on some customers’ environments.

• At the same time, as seen above, both time and money dispenses must be reduced to the minimum, and this without affecting the marketing aspect so without affecting the features allowing to be part of the targeted market leaders.

How would you combine these two apparently contradictory requirements?

We will see a few techniques allowing to achieve this.

Take design shortcuts

The examples we saw at the beginning of this article are perfect illustrations of this:

• The mentioned editors could have implemented a proper authentication framework for inter-components communication. This would require design, coding, debugging: all the long and tedious steps I’ve just mentioned.

  There is neither time nor money to spend on such low-visibility features.

• Or they can simply create a central hardcoded account and provide it administrative accesses to avoid any privilege-related issue.

  Problem “solved”.

Of course, this is a disaster in terms of security: it profoundly undermine the security of their customers and usually, as a chain effect, the security of their own customers too and so on, but who cares?

As I said earlier, most chances are that nobody will ever find this anyway, and the expected benefits by far outweigh the potential consequences of having to issue a fix and orchestrate a few marketing gimmicks once the market has been acquired.

Don’t test the code

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. — Brian Kernighan

Debugging is one the most complex and tedious step in software engineering. A huge economy can be made by just bypassing the testing and debugging phases on non-critical features. At most, test against a handful of well-defined stories allowing to produce working demos for the sales forces , but don’t bother with anything else.

Most probably these features will be unusable in any real-life situation, but this doesn’t matter. The goal is to sell a product, and to sell it it must be available. The fact that it doesn’t really work can be handled later, as part of the maintenance process, once the client already paid for the product and its support contract.

Advertise but don’t implement the feature at all

Just put some stubs and hope for the best.

This approach is particularly useful for secondary features, required to be able to advertise a complete solution but not used by any current or targeted key customers².

Would a consumer try to use that feature, it will obviously not work, but as described in the previous point this can be officially handled as a “bug”, as if the feature was there but the customer encountered an unexpected error. The “fix” will then be to implement the missing feature, or more rationally to implement the parts of the feature necessary for the customer’s use-case.

From the customer point-of-view, such situation is normally indistinguishable from a normal bug and the editor won’t communicate about the feature being actually missing for obvious reasons. However, such shortcut can be seen in more open projects, such as Android where important functions such as gethostbyname() were still left as stub in the version 2.2 (Android Froyo, 2010):

struct hostent ∗gethostbyname(const char ∗name) {
    printf("FIX ME! implement %s %s:%d\n", __FUNC__, __file__,__line__);
    return NULL;
}

This example takes a system API function, but the same principle is also often used for more high-level functionalities considered secondary in terms of ROI.

The people who use them

Another thing often missed is that the software quality is also heavily influenced by the internal organization withing the software company.

Let’s take two examples, at the technical and the management layers.

Development and maintenance teams

Some companies distinguish:

• The development teams, who create the next generation software. They work before the release of a software.

• The maintenance teams, who address bugs as they are reported by users. They work after a software has been released.

When there is a lack of involvement (and often the combined pressure of short deadlines, low budget and ever-changing requirements quickly weakens it), the development team may have a tendency to release poor quality and buggy code to handover the debugging process to the maintenance team.

In other words, they limit themselves to the most interesting and gratifying part of the job, and leave the rest to the maintenance team, effectively releasing unfinished software to the customers.

Management turnover

Some companies tend to increase internal turnover in management positions notably as an attempt to increase the company’s global reactivity.

It can be tempting in such situation, for a project manager, to take decisions which make no sense in the long run but allow him to present better metrics to his upper-management (or in an resume if he foresees a company change). The bet is that by the time his house of cards collapses he will already be promoted to another position and leave his successor to handle the consequences.

And how they use them

These techniques are tools, and like any tool they are not bad by themselves: there are only good and bad usages.

For instance, it is an accepted practice to release a software with known bugs considered non-blocking. There is indeed no point in penalizing the whole user-base by delaying a release because of a bug affecting an obscure legacy feature present only for backward compatibility and in fact not used by anyone anymore.

There is no such thing as bug-free software, in particular for large and complex software such as the ones used in corporate environments, as highlighted by the following XKCD comic:

But that’s not to say that editors should satisfy themselves in delivering burning buses to their customers:

Conclusion

Large software editors should not forget their social contract: they are software editors, not investment funds. A company is meant to be a production unit, delivering either goods or services to the society, and getting money in return.

There are and will always be ambitious and selfish people trying to circumvent the system and get the money without really delivering anything in the first place, but the society also develops safeguards to make such behavior more an exception than the rule.

Public disclosure allows to inform the society as a whole when issues such as the ones mentioned in the beginning of this article are found. When such issues come too often its the editor own credibility which takes a hit, despite its investment in marketing and public relations.

Moreover, at the same time free software is raising and can be found at more and more places in corporate environments. While free software projects suffer in their first stages from their relative slow development pace compared to large company reactivity, in the long run this actually gives them an advantage over most proprietary software.

As a consequence of their need of reactivity, software companies develop short roadmaps. A feature announced this year and motivating a customer to invest on a software may very well be abandoned three years laters due to market changes (remember the Magic Quadrant mentioned earlier?), generating a pure-loss on customer’s side.

Free software provides several guaranties:

• The source-code is public and auditable.

  While as any software it may still contain security bugs, development mistakes with security repercussions, any large project will contain virtually no backdoor, meaning no code developed with the sole and official intent of bypassing security measures.

• Free software projects are usually users-centered, not investors-centered.

  They don’t have the requirements imposed by the profitability race, providing better long-term visibility to the users base.

• Users can participate or take-over a project development, leaving the traditional customer-supplier relationship and becoming directly involved in the product design and development.

  This can take various shape:

  • Participating directly to the project by dedicating some internal development resources. This effort can optionally be coordinated for instance by founding associations such as the newly created French TOSIT which gather several large companies and allow them to evenly spare the efforts and avoid duplicated work.

  • Paying external developers to implement wanted features, either developers already belonging to the project or a independent IT company providing Free Software development services.

  • By forking the project, either as a private (if the license allows it) or public fork.

    This is not exclusive with the previous bullets, and a good example can be Android which has numerous active forks maintained by various entities, both in the private and the public sector.

With large software editors becoming less and less an obligatory step, we can hope that they will perceive software quality as a more important criterion, if not for customer’s or society’s sake, at least to avoid reputation and sell loss with the financial repercussion they will have.

If not, then too bad for them. I think that such issues are just the best advertisement that they could make to engage people toward FOSS.