Hello all,

Please find the test report of oFono v1.13 commit 7a6da06.

During this testing, we ran 34 functional positive cases. 32 cases passed, 1 failed and 1 blocked due to bug (see Issue Summary section for details). The Pass Rate is 94% (-6% vs previous session). One regression has been found concerning Lock/Unlock PIN code feature.

Specifically for 1.13 release which supports HFP 1.6 features, a PTS test session has been also run to validate it. We ran a total of 75 functional positive cases. 72 cases passed, 0 failed and 3 blocked due to issue (see Notes section for details) or feature not supported. The Pass Rate is 96%.

If you have any comment about this report, tell me please.

-----------------------------------------------

Test Objective

The aim of this session was to validate the state of oFono upstream by testing only major tests of most important features over all material we had (according to what feature was supported and by priority order: PR3, Network Simulator, 3G dongle) and specifically for 1.13 release which supports HFP 1.6 features a PTS test session has been also run to validate it. oFono has been installed and tested on PR3 running Tizen 2.0 and Ubuntu 12.04 device for standard Telephony features and installed on IVI board for HFP features.

-----------------------------------------------

Test Environment

For all Setup:
oFono: v1.13 (updated to commit 7a6da06)

 -      PR3 Setup:
    Image: Tizen_Mobile_ofono_branch_09_06_2013 (Tizen 2.0 image)
    Operator & SIM Card: SFR SIM Card – Phone number +33623312183, Bouygues SIM Card – Phone number +33760660848

 -      Laptop + 3G Dongle/Phonesim:
usb_modeswitch: v1.2.5
modeswitch data: 20120120
Hardware: Laptop
Ubuntu: v12.04
Modem: Huawei E173u-2 - Operator & SIM Card: SFR SIM Card – Phone number +33623312183, Bouygues SIM Card – Phone number +33663999052

    -      IVI Setup (for PTS tests)
    Device: Ndis 166 IVI board
    Image: tizen_20131030.3_ivi-mbr-i586-sdb
    Packages installed (which was missing in the image):
        bluez-cups-5.10-1.1.i686.rpm
        bluez-devel-5.10-1.1.i686.rpm
        bluez-docs-5.10-1.1.i686.rpm
        bluez-test-5.10-1.1.i686.rpm
        libpulse-devel-4.0-4.1.i686.rpm
        ofono-devel-1.13-1.i686.rpm (updated to commit 7a6da06)
        ofono-docs-1.13-1.i686.rpm (updated to commit 7a6da06)
        ofono-test-1.13-1.i686.rpm (updated to commit 7a6da06)
        pulseaudio-docs-4.0-4.1.i686.rpm
        pulseaudio-module-augment-properties-4.0-4.1.i686.rpm
        pulseaudio-module-combine-sink-4.0-4.1.i686.rpm 
        pulseaudio-module-dbus-protocol-4.0-4.1.i686.rpm
        pulseaudio-module-devel-4.0-4.1.i686.rpm
        pulseaudio-module-filter-4.0-4.1.i686.rpm
        pulseaudio-module-null-source-4.0-4.1.i686.rpm
        pulseaudio-module-raop-4.0-4.1.i686.rpm
        pulseaudio-module-switch-on-connect-4.0-4.1.i686.rpm
        pulseaudio-vala-bindings-4.0-4.1.i686.rpm

-----------------------------------------------

Issue Summary

New bug: 2

OF-168 - Unable to lock/unlock PIN Code while modem is online
https://01.org/jira/browse/OF-168

Issue: Unable to initiate audio connection (error in btmon when executing "set-card-profile x hfgw" pacmd: Unsupported Remote Feature / Unsupported LMP Feature (0x1a))
Components: To be analyzed (hardware issue?)
Severity: Normal
Status: Open
Assignee: Luiz Von Dentz

Known bug: 1

OF-162 - Going back from a select item list releases the session
https://01.org/jira/browse/OF-162

Closed bug: 0
None

-----------------------------------------------

Test Result

Result Summary – Basic Features

Total Test Case

34

Passed

32

Failed

1

Blocked

1

TCs completed

100,0%

Run rate

97%

Pass rate total

94%

Blocked rate total

3%

Pass rate of executed

97%

Test Result by feature

Features (and their status in color)

Total

Pass

Fail

Blocked

Pass %

Modem Used

Modem

5

5

0

0

100%

Tested on smartphone using a real SIM or on Laptop using 3G Dongle

SIM

4

2

1

1

50%

Tested on Laptop using 3G Dongle

Network

2

2

0

0

100%

Tested on smartphone using a real SIM

Connectivity

8

8

0

0

100%

Tested on smartphone using a real SIM or on Laptop using 3G Dongle

Voice Calls

12

12

0

0

100%

Tested on smartphone with appropriate SIM Card and Laptop + HFP connected with a smartphone

Messaging

2

2

0

0

100%

Tested on smartphone using a real SIM

Message Waiting

1

1

0

0

100%

Tested on smartphone using a real SIM


Please find details in the attached file.

Result Summary – PTS HFP 1.6 Features

Total Test Case

75

Passed

72

Failed

0

Blocked

3

TCs completed

100,0%

Run rate

96%

Pass rate total

96%

Blocked rate total

4%

Pass rate of executed

100%

Test Result by feature

Features (and their status in color)

Total

Pass

Fail

Blocked

Pass %

Transfer of Registration Status

1

1

0

0

100%

Phone status information

4

4

0

0

100%

Audio Connection handling

4

4

0

0

100%

Audio Connection release

2

2

0

0

100%

Accept an incoming voice call

1

1

0

0

100%

Incoming voice call audio scenarios

6

6

0

0

100%

Reject an incoming voice call

2

2

0

0

100%

Terminate a voice call

4

4

0

0

100%

Audio Connection transfer during an ongoing call to HF

5

3

0

2

60%

Audio Connection transfer during an ongoing call to AG

2

2

0

0

100%

Place a call with the phone number supplied by the HF

1

1

0

0

100%

Call waiting notification - Three way calling

6

6

0

0

100%

Incoming voice call interrupted by remote

1

1

0

0

100%

Echo canceling (EC) and noise reduction (NR)

1

1

0

0

100%

Attach a phone number to a voice tag

1

0

0

1

0%

Ability to transmit DTMF codes

1

1

0

0

100%

Remote audio speaker volume control

3

3

0

0

100%

Remote audio micro volume control

3

3

0

0

100%

Query current call status

3

3

0

0

100%

Enhanced Call Control

2

2

0

0

100%

Service Level Connection establishment - Codec Negotiation

7

7

0

0

100%

Codec Connection set up

7

7

0

0

100%

Wide Band Speech

2

2

0

0

100%

Connection management

2

2

0

0

100%

Connection management - Service Search Request

3

3

0

0

100%

Place a call with the phone number supplied by the AG

1

1

0

0

100%


-----------------------------------------------

Notes

Please note that one oFono crash still occurs on the following conditions:
-      OF-163 - oFono crashes when doing a NAA Initialization+File Change Notification after having done a NAA Initialization+Full File Change Notification with Phonesim.

A bug in PTS software (11515 which sends to oFono the wrong HFP version supported) preventing to execute several TCs has been avoided during test session.
    Workaround details:
        1.       Change the HFP version in /var/lib/Bluetooth/<Local mac address>/cache/<PTS mac address>
        2.       Kill Bluetoothd
        3.       Relaunch Bluetoothd
        4.       Redo the PTS tests that was blocked by this issue.


Best regards,
Nicolas