General Focus and Position
In most cases, the development approach used
for Web-based systems has been ad hoc, and software development for the Web
lacks rigour and a systematic approach. Hardly any attention is given to
development methodologies, measurement and evaluation techniques,
application quality, reliability, and project management. Further, most
current applications development and management practices heavily rely on
the knowledge and experience of individual developers and their own
development practices. In addition, there is lack of proper testing of
Web-based systems, and documentation which is needed for ‘maintenance and
upgrade’ of the system among other needs.
The nature and rapid growth/evolution of the
Web, the rapid boom in Web and Web-related technologies, the
commercialisation of the Web, and the rush to "be on the Web" and
to migrate legacy systems to Web environments can partly be attributed to
problems of Web-based system development. Also the complexity of Web-based
applications has grown significantly - from information dissemination
(consisting of simple text and images to image maps, forms, CGI, applets,
scripts and stylesheets) to online transactions, enterprise-wide planning
and scheduling systems, Web-based collaborative work environments, etc.
While the focus in SPIQ was on more long-term
measurement based software process improvement (SPI), PROFIT focuses more
on SPI in turbulent environments. However, both SPIQ and PROFIT focused on
more traditional software development. WEBSYS differs from SPIQ and
PROFIT in the explicit focus on the development of Web-based systems and
the extremely short life cycles and corresponding extreme requirements of
reduced time to market. Furthermore, Web’s legacy as an information medium
rather than an application medium is another cause of the problem. Many
consider Web development primarily as an authoring problem rather than an
application development problem to which some of the well-known software
engineering and management principles and practices could apply – with some
changes and tuning to suite to the Web environment.
Many attributes of quality Web-based Systems
such as ease of navigation, accessibility, scalability, maintainability,
usability, compatibility and interoperability, security, readability, and
reliability are not given due consideration during development. Many
developers seem to be unaware of the real issues and challenges facing
major Web-based application development and its continual maintenance.
There is a need to engender an awareness of
the need for more disciplined approaches to Web-based application
development, and we need to move from the current, largely ad hoc (and
personalised) approach to a better disciplined approach and process. Also
we need to realise that Web-based system development is not just graphic
design or content development any more, and there are growing number of
complex applications – intranet-based applications, transactional systems,
and other e-business applications. Hence, Web-based systems developments
are becoming more like software projects, and less like work of art.
Web Engineering (WebE) is an emerging new
discipline focusing on successful development of complex web-based systems
and applications (Ref ICSE99 workshop). It is concerned with establishment
and use of sound scientific, engineering and management principles and
disciplined and systematic approaches to the successful development,
deployment and maintenance of high quality Web-based systems and
applications. Thus, we see a close relationship between Web Engineering,
Software Reliability and Software Process Improvement.
Software reliability is defined as the
probability that the software will fail in a certain interval under stated
operational conditions. For a web site there are two factors that are
important - robustness and availability.
Robustness is the system’s ability to
keep on working even if it gets erroneous or unexpected input. For a web
system, diverse, wrong or unexpected input is more the rule than the
exception. Thus, robustness is of big importance here.
Availability is the probability
that the system is available on demand. It depends on the system’s
reliability and capability for recovery. For most users, availability will
reflect the site’s quality of service. A high availability will be an
important factor in a competitive environment.
Both these factors taken together are
necessary for a stable application and good service. The diversity of
operational platforms where the applications will run puts an extra load on
the system, especially when it comes to robustness. It is important to bear
in mind that if a web site is down, the owner most likely will loose
business since a prospective user just will move on to the next site which
offer the same or a comparable service.
There exist several techniques and methods
that can be used to achieve reliability. See for instance
[Stålhane98a, 98b]. Most of them are, however, mainly used in
military systems and systems related to air and rail traffic. By modifying
and carrying these techniques and methods over to web software we can obtain
a considerable increase in system’s reliability. In addition, by
identifying the impact of each technique and method and describe them in
quantitative terms, we can offer a way of estimating the effect of
including or leaving them out of the development process. See for instance
Further, the problems with changing the
process, introducing new technology and in demonstrating their effects, are
shared with SPI (Software Process Improvement). For instance, there
may be no clear baseline to compare with, results may come late or are
unconvincing, the current product delivery suddenly gets priority and all
"experimentation" is postponed or forgotten, management is
semi-committed, or too dependent on local champions. In addition comes the usual
organisational and technological turbulence in a highly evolving business
All WEBSYS partners are or have been
involved in the industrial, NFR-supported SPIQ [Ulsund96] and PROFIT
[Ulsund99] projects in, respectively, 1997-99 and 2000-02. In both SPIQ and
PROFIT, about 10 companies have been or are participating – and with
SINTEF, NTNU and UiO as research partners. In SPIQ, a downscaled and
pragmatic SPI framework was defined as part of the SPIQ Method Handbook (in
Norwegian) [Dybå00a]. The Handbook attempts to combine TQM ideas with
QIP/EF/GQM. The partners have participated in more than 20 such SPI efforts
in the software industrial during the last 3.5 years.
Thus, the personnel that will work in the WEBSYS
project have experience and knowledge related to the areas of Web-based
systems development, software reliability, and software process
improvement. The partners have more than 20 international publications so
far in 2000 based on empirical work in Norwegian industry.
All three research partners have emphasised
empirical studies in their former work on software process improvement and
on software engineering in general, cf. the SPIQ Method Handbook
[Dybå00a]. We have also been working on guidelines for empirical
studies [Arisholm99a] – see also other references to own work in the
previous section. In WEBSYS we will keep up our focus on performing
solid empirical studies – including formal (controlled) experiments, case
studies, interviews and observations (think-aloud protocols, etc.) and
The two most relevant research methods for
studies on Web-based systems development are variants of observation and
experimentation, see e.g. [Yin94] and [Wohlin00]. Indeed, a new research
field, Empirical Software Engineering, is emerging. This is
especially due to the work of Victor Basili in Maryland and Hans-Dieter
Rombach in Kaiserslautern, both with applied research institutes
affiliated, and a long shared history for using the NASA-SEL as a
Observation of real software development work
has the advantage of being realistic regarding scale and environment, but
there are difficulties due to lack of control of variables, i.e.
cause-effect analyses are hard. A spectrum of empirical techniques are
available, e.g. formal experiments, case studies, surveys, retrospective
analysis, even literature studies etc. [Zelkowitz98].
Experiments may have control of variables,
but have shortcomings with respect to realism, e.g. unrealistic small
software development tasks. We therefore need a combination: To establish
reasonable, validated results, we should both carry out research in a
controlled environment (as realistically as possible) and in a realistic
environment (as controlled as possible). For projects like WEBSYS, it is
advisable to combine observation of real projects (often case studies) with
surveys and "laboratory" experiments (often on own students).
We have carried out a number of observational
and experimental studies on software development where results on research
method have been a major component (see references in Sections 5.1-5.3).
UiO has published some papers on research methods [Arisholm99a] [Anda00]
and [Jørgensen00], and been responsible for several seminars on
research methods. In addition, UiO has implemented a measurement program in
a large Norwegian software company to assess the impact of incremental
development on productivity, time-to-market and delivery precision.
UiO has been working closely with prof.
Welland and prof. Atkinson at Glasgow for many years on empirical studies
of database schema evolution and the application of persistent language
technology. Recently, UiO has also been working with Dr. Lionel Briand at
Carleton University, Ottawa on empirical evaluation of metrics for object-oriented
NTNU has for five years used own students in
running experiments in a software quality course [Jaccheri00]. It has run
many case studies in co-operation with industry, see e.g. the one project
evolution [Nguyen97] and on inspections in Ericsson [Conradi99] as part of
the SPIQ project. There is also ongoing, joint work with University of
Maryland on how to represent repeatable experiments, and in running
repeatable experiments on own students [Conradi+00a]. Much empirical work,
especially around testing and SPI, has also been done at SINTEF by future
NTNU-colleague Tor Stålhane, see [Haugrud98],
[Stålhane97], and by Tore Dybå, see Dybå[97, 99, 00b,
The planned guest researchers/postdocs may
come from the collegial environments mentioned above, e.g. from Maryland,
Kaiserslautern,Univ. of London, Politecnico di Milano, Univ. Lund.
Collaboration between Academia and
will be conducted in close co-operation between the academic partners and
the industrial partners to give a strong focus on practical application of
the methods and empirical feedback. Much of the academic work will be done
as part of PhD degrees, where each candidate typically will work with one
or two of the participating companies. The candidates will gather valuable
practical experience and empirical data to test the research hypotheses of
their theses in a sound way.
companies will achieve better understanding and improvement of their own
software development processes, and base the development of their Web-based
tools on this increased competence and the results of WEBSYS and,
hence, increase their competitiveness.
We will also
continue our tradition in using MSc students to work together with
researchers and PhD students in such empirical work. In addition, each
partner has 3-4 courses where more controlled experiments have been carried
out for more than 5 years.
intends to create and utilise a synergy with related projects. In addition
to working together with software producers on concrete cases, technology
transfer to Norwegian organisations will be ensured through the
dissemination activities described in the work plan.