| Press Release on September 11, 2012 |
|
| Abstract | Latest Figures |Commentary | Future Release (Tentative) |
| Detailed Findings (Popularity of Chief Executive/ Popularity of Principal Officials/Public Sentiment Index Feature Page) |
|
Abstract
POP interviewed 1,005 Hong Kong people between September 3 to 7, 2012 by means of a random telephone survey conducted by real interviewers. Our latest survey conducted before the Legislative Council Election shows that the popularity of CE CY Leung continues to drop, with support rating of 46.5 marks and approval rate of 33%. Both figures are at their record lows since he was elected as the CE. His net popularity now stands at negative 17 percentage points. As for the Secretaries of Departments, the latest support rating of CS Carrie Lam is 58.1 marks, her approval rate is 55%, disapproval rate 14%, giving a net popularity of positive 41 percentage points. Latest support rating of FS John Tsang is 54.3 marks, approval rate 53%, disapproval rate 12%, giving a net approval rate of positive 41 percentage points. As for SJ Rimsky Yuen, his support rating is 49.1 marks, approval rate 24%, disapproval rate 6%, and his net popularity stands at positive 18 percentage points. Carrie Lam remains to be the most popular Secretary of Department. As for the Directors of Bureaux, compared to one month ago, the net approval rate of 5 of the 12 Directors have gone up while 7 have gone down. Among them, Secretary for Environment Wong Kam-sing, Secretary for Security Lai Tung-kwok, Secretary for Labour and Welfare Matthew Cheung, Secretary for Financial Services and the Treasury Ceajer Chan and Secretary for the Civil Service Paul Tang have registered changes in net approval rates beyond sampling error, up by 10, 9 and down by 9, 7 and 7 percentage points respectively. Among all the Directors, only Secretary for Development Paul Chan and Secretary for Education Eddie Ng register negative popularity, at negative 29 and 32 percentage points respectively. Secretary for Food and Health Ko Wing-man continues to be the most popular Director. According to POP’s standard, Ko Wing-man falls under the category of “ideal” performer, Matthew Cheung, Carrie Lam, Lai Tung-kwok and John Tsang now fall under the category of “successful” performer. The performance of Anthony Cheung, Tsang Tak-sing, Eddie Ng and Paul Chan can be labeled as “mediocre”, and that of Ceajer Chan, Raymond Tam, Wong Kam-sing, Gregory So, Rimsky Yuen and Paul Tang can be labeled as “inconspicuous”. CY Leung can be labeled as “depressing” while no one falls into the category of “disastrous”. As for people’s public sentiment, our Public Sentiment Index (PSI) score continues to drop, both in terms of Government Appraisal (GA) and Society Appraisal (SA) scores, meaning that the public sentiment has turned worse. The maximum sampling error of all approval and disapproval rates is +/-4 percentage points at 95% confidence level, while the sampling error of rating figures and net approval rates needs another calculation. The response rate of the popularity survey of officials is 65%.
Points to note:
[1] The address of the "HKU POP SITE" is http://hkupop.hku.hk, journalists can check out the details of the survey there.
[2] The sample size of this survey is 1,005 successful interviews, not 1,005 x 64.6% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of all percentages of each survey is +/-4 percentage points at 95% confidence level, while the sampling error of rating figures and net approval rates needs another calculation. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state "sampling error of various ratings not more than +/-2.2, sampling error of percentages not more than +/-4% and sampling error of net approval rates not more than +/-7% at 95% confidence level".
[4] When quoting percentages of this survey, journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used, in order to match the precision level of the figures.
[5] The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.
|
Latest Figures
I. Popularity figures of CE CY Leung and principal officials
POP today releases the latest popularity figures of CE CY Leung and various Secretaries of Departments and Directors of Bureaux under the accountability system. All the figures have been weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in mid-year 2012. Herewith the contact information for the latest survey:
Date of survey |
Overall sample size |
Response rate |
Maximum sampling error of percentages[6] |
3-7/9/2012 |
1,005 |
64.6% |
+/-3% |
[6] Errors are calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Questions using only sub-samples would have bigger sampling error. Sampling errors of ratings and net approval rates are calculated according to the distribution of the scores collected.
As different questions involve different sub-samples, the sampling errors will vary accordingly. The table below briefly shows the relationship between sample size and maximum sampling errors for the readers to capture the corresponding changes:
Sample size
(total sample or sub-sample) |
Sampling error of percentages[7]
(maximum values) |
Sample size
(total sample or sub-sample) |
Sampling error of percentages[7]
(maximum values) |
1,300 |
+/- 2.8 % |
1,350 |
+/- 2.7 % |
1,200 |
+/- 2.9 % |
1,250 |
+/- 2.8 % |
1,100 |
+/- 3.0 % |
1,150 |
+/- 3.0 % |
1,000 |
+/- 3.2 % |
1,050 |
+/- 3.1 % |
900 |
+/- 3.3 % |
950 |
+/- 3.2 % |
800 |
+/- 3.5 % |
850 |
+/- 3.4 % |
700 |
+/- 3.8 % |
750 |
+/- 3.7 % |
600 |
+/- 4.1 % |
650 |
+/- 3.9 % |
500 |
+/- 4.5 % |
550 |
+/- 4.3 % |
400 |
+/- 5.0 % |
450 |
+/- 4.7 % |
[7] Based on 95% confidence interval.
Recent popularity figures of CE CY Leung are summarized as follows:
Date of survey |
19-25/6/12 |
9-11/7/12 |
17-20/7/12 |
2-8/8/12 |
14-18/8/12 |
3-7/9/12 |
Latest change |
Sample base |
1,048 |
1,000 |
1,018 |
1,002 |
1,019 |
1,005 |
-- |
Overall response rate |
69.6% |
69.1% |
65.8% |
65.9% |
66.6% |
64.6% |
-- |
Finding |
Finding |
Finding |
Finding |
Finding |
Finding |
Finding and error[8] |
-- |
Rating of CE CY Leung |
51.3[9] |
53.8[9] |
51.2[9] |
51.6 |
49.0[9] |
46.5+/-1.6 |
-2.5[9] |
Vote of confidence in CE CY Leung |
47% |
45% |
41%[9] |
39% |
36% |
33+/-3% |
-3% |
Vote of no confidence in CE CY Leung |
42% |
41% |
45%[9] |
47% |
48% |
50+/-3% |
+2% |
Net approval rate |
5% |
4% |
-4%[9] |
-8% |
-12% |
-17+/-6% |
-5% |
[8] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Media can state "sampling error of rating not more than +/-1.6, sampling error of percentages not more than +/-3%, sampling error of net approval rates not more than +/-6% at 95% confidence level" when quoting the above figures. The error margin of previous survey can be found at the POP Site.
[9] Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
Recent popularity figures of the three Secretaries of Departments under the accountability system are summarized below:
Date of survey |
3-10/5/12 |
30/5-6/6/12 |
9-11/7/12 |
2-8/8/12 |
3-7/9/12 |
Latest change [10] |
Sample base[10] |
515-538 |
521-566 |
529-572 |
523-563 |
577-631 |
-- |
Overall response rate |
64.6% |
63.7% |
69.1% |
65.9% |
64.6% |
-- |
Latest finding |
Finding |
Finding |
Finding |
Finding |
Finding & error [11] |
-- |
Rating of CS Carrie Lam |
-- |
-- |
64.0 |
62.0[12] |
58.1+/-1.8 |
-3.9[12] |
Vote of confidence in CS Carrie Lam |
-- |
-- |
60% |
59% |
55+/-4% |
-4% |
Vote of no confidence in CS Carrie Lam |
-- |
-- |
8% |
11% |
14+/-3% |
+3% |
Net approval rate |
-- |
-- |
52% |
48% |
41+/-6% |
-7%[12] |
Rating of FS John Tsang |
52.7[12] |
52.9 |
56.4[12] |
58.1[12] |
54.3+/-1.6 |
-3.8[12] |
Vote of confidence in FS John Tsang |
43%[12] |
44% |
49%[12] |
54%[12] |
53+/-4% |
-1% |
Vote of no confidence in FS John Tsang |
19%[12] |
16% |
16% |
13% |
12+/-3% |
-1% |
Net approval rate |
24% |
28% |
33% |
41%[12] |
41+/-6% |
-- |
Rating of SJ Rimsky Yuen |
-- |
-- |
49.0 |
50.6 |
49.1+/-2.2 |
-1.5 |
Vote of confidence in SJ Rimsky Yuen |
-- |
-- |
24% |
23% |
24+/-4% |
+1% |
Vote of no confidence in SJ Rimsky Yuen |
-- |
-- |
12% |
8%[12] |
6+/-2% |
-2% |
Net approval rate |
-- |
-- |
12% |
15% |
18+/-4% |
+3% |
[10] The frequency of this series of questions is different for different questions, and also different from that of CE popularity ratings. Comparisons, if made, should be synchronized using the same intervals. Starting from 2011, these questions only uses sub-samples of the tracking surveys concerned, the sample size for each question also varies.
[11] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Media can state "sampling error of various ratings not more than +/-2.2, sampling error of percentages not more than +/-4%, sampling error of net approval rates not more than +/-6% at 95% confidence level" when quoting the above figures. The error margin of previous survey can be found at the POP Site.
[12] Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
Latest popularity figures of Directors of Bureaux under the accountability system are summarized below, in descending order of net approval rates:
Date of survey |
9-11/7/12 |
2-8/8/12 |
3-7/9/12 |
Latest change |
Sample base [13] |
528-606 |
514-588 |
551-643 |
-- |
Overall response rate |
69.1% |
65.9% |
64.6% |
-- |
Sample base for each question/ Percentage of answer |
Base |
% |
Base |
% |
Base |
% & error [14] |
-- |
Vote of confidence in Secretary for Food and Health Ko Wing-man |
528 |
69% |
531 |
67% |
607 |
69+/-4% |
+2% |
Vote of no confidence in Secretary for Food and Health Ko Wing-man |
528 |
5% |
531 |
5% |
607 |
3+/-1% |
-2% |
Net approval rate |
-- |
64% |
-- |
62% |
-- |
66+/-4% |
+4% |
Vote of confidence in Secretary for Labour and Welfare Matthew Cheung |
532 |
64%[15] |
538 |
65% |
615 |
55+/-4% |
-10%[15] |
Vote of no confidence in Secretary for Labour and Welfare Matthew Cheung |
532 |
10% |
538 |
8% |
615 |
7+/-2% |
-1% |
Net approval rate |
-- |
54%[15] |
-- |
57% |
-- |
48+/-5% |
-9%[15] |
Vote of confidence in Secretary for Security Lai Tung-kwok |
542 |
55% |
527 |
45%[15] |
619 |
54+/-4% |
+9%[15] |
Vote of no confidence in Secretary for Security Lai Tung-kwok |
542 |
11% |
527 |
11% |
619 |
11+/-2% |
-- |
Net approval rate |
-- |
44% |
-- |
34%[15] |
-- |
43+/-5% |
+9%[15] |
Vote of confidence in Secretary for Financial Services and the Treasury Ceajer Chan |
584 |
48%[15] |
531 |
44% |
551 |
38+/-4% |
-6%[15] |
Vote of no confidence in Secretary for Financial Services and the Treasury Ceajer Chan |
584 |
8%[15] |
531 |
6% |
551 |
7+/-2% |
+1% |
Net approval rate |
-- |
40%[15] |
-- |
38% |
-- |
31+/-5% |
-7%[15] |
Vote of confidence in Secretary for Transport and Housing Anthony Cheung |
540 |
46% |
514 |
41%[15] |
643 |
41+/-4% |
-- |
Vote of no confidence in Secretary for Transport and Housing Anthony Cheung |
540 |
11% |
514 |
12% |
643 |
14+/-3% |
+2% |
Net approval rate |
-- |
35% |
-- |
29% |
-- |
27+/-5% |
-2% |
Vote of confidence in Secretary for Constitutional and Mainland Affairs Raymond Tam |
557 |
37%[15] |
542 |
32%[15] |
578 |
36+/-4% |
+4% |
Vote of no confidence in Secretary for Constitutional and Mainland Affairs Raymond Tam |
557 |
15% |
542 |
14% |
578 |
13+/-3% |
-1% |
Net approval rate |
-- |
22% |
-- |
18% |
-- |
23+/-6% |
+5% |
Vote of confidence in Secretary for the Environment Wong Kam-sing |
562 |
37% |
542 |
28%[15] |
605 |
33+/-4% |
+5%[15] |
Vote of no confidence in Secretary for the Environment Wong Kam-sing |
562 |
9% |
542 |
15%[15] |
605 |
10+/-2% |
-5%[15] |
Net approval rate |
-- |
28% |
-- |
13%[15] |
-- |
23+/-5% |
+10%[15] |
Vote of confidence in Secretary for Commerce and Economic Development Gregory So |
549 |
34%[15] |
561 |
26%[15] |
598 |
26+/-4% |
-- |
Vote of no confidence in Secretary for Commerce and Economic Development Gregory So |
549 |
12%[15] |
561 |
14% |
598 |
13+/-3% |
-1% |
Net approval rate |
-- |
22%[15] |
-- |
12%[15] |
-- |
13+/-5% |
+1% |
Vote of confidence in Secretary for the Civil Service Paul Tang |
541 |
24% |
588 |
24% |
635 |
20+/-3% |
-4% |
Vote of no confidence in Secretary for the Civil Service Paul Tang |
541 |
7% |
588 |
6% |
635 |
9+/-2% |
+3%[15] |
Net approval rate |
-- |
17% |
-- |
18% |
-- |
11+/-4% |
-7%[15] |
Vote of confidence in Secretary for Home Affairs Tsang Tak-sing |
558 |
37%[15] |
530 |
37% |
557 |
34+/-4% |
-3% |
Vote of no confidence in Secretary for Home Affairs Tsang Tak-sing |
558 |
31% |
530 |
24%[15] |
557 |
28+/-4% |
+4% |
Net approval rate |
-- |
6%[15] |
-- |
13% |
-- |
6+/-7% |
-7% |
Vote of confidence in Secretary for Development Paul Chan[16] |
-- |
-- |
529 |
17% |
619 |
16+/-3% |
-1% |
Vote of no confidence in Secretary for Development Paul Chan[16] |
-- |
-- |
529 |
42% |
619 |
45+/-4% |
+3% |
Net approval rate |
-- |
-- |
-- |
-25% |
-- |
-29+/-6% |
-4% |
Vote of confidence in Secretary for Education Eddie Ng |
606 |
33% |
527 |
20%[15] |
601 |
17+/-3% |
-3% |
Vote of no confidence in Secretary for Education Eddie Ng |
606 |
20% |
527 |
48%[15] |
601 |
49+/-4% |
+1% |
Net approval rate |
-- |
13% |
-- |
-28%[15] |
-- |
-32+/-6% |
-4% |
[13] Starting from 2006, these questions only uses sub-samples of the tracking surveys concerned, the sample size for each question also varies.
[14] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Media can state "sampling error of percentages not more than +/-4% and sampling error of net approval rates not more than +/-7% at 95% confidence level" when quoting the above figures. The error margin of previous survey can be found at the POP Site.
[15] Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
[16] The government announced the appointment of Paul Chan as the new Secretary for Development on July 30, 2012.
The latest survey showed that, CE CY Leung scored 46.5 marks, and 33% supported him as CE, his net approval rate is negative 17 percentage points. Meanwhile, the corresponding ratings of CS Carrie Lam, FS John Tsang and SJ Rimsky Yuen were 58.1, 54.3 and 49.1 marks, and 55%, 53% and 24% would vote for their reappointment correspondingly. Their net approval rates are positive 41, 41 and 18 percentage points respectively.
As for the Directors of Bureaux, according to the net approval rates, results revealed that the top position goes to Secretary for Food and Health Ko Wing-man, attaining positive 66 percentage points. The 2nd to 4th places belonged to Secretary for Labour and Welfare Matthew Cheung, Secretary for Security Lai Tung-kwok and Secretary for Financial Services and the Treasury Ceajer Chan with net approval rates positive 48, 43 and 31 percentage points respectively. Secretary for Transport and Housing Anthony Cheung, Secretary for Constitutional and Mainland Affairs Raymond Tam, Secretary for the Environment Wong Kam-sing, Secretary for Commerce and Economic Development Gregory So, Secretary for the Civil Service Paul Tang, Secretary for Home Affairs Tsang Tak-sing, Secretary for Development Paul Chan and Secretary for Education Eddie Ng ranked 5th to 12th, their corresponding net approval rates are positive 27, positive 23, positive 23, positive 13, positive 11, positive 6, negative 29 and negative 32 percentage points. In other words, Ko Wing-man scored net approval rates of over 50% among all Directors of Bureaux.
II. PSI analysis
The Public Sentiment Index (PSI) [17] compiled by POP aims at quantifying Hong Kong people’s sentiments, in order to explain and predict the likelihood of collective behaviour. PSI comprises 2 components: one being Government Appraisal (GA) Score and the other being Society Appraisal (SA) Score. GA refers to peoples’ appraisal of society’s governance while SA refers to peoples’ appraisal of the social environment. Both GA and SA scores are compiled from a variety of opinion survey figures. All PSI, GA and SA scores range between 0 to 200, with 100 meaning normal. Please refer to our online material for its methodology. Latest PSI, GA & SA figures, per poll, are tabulated as follow:
Date of survey (per poll) |
9-11/7/12 |
17-20/7/12 |
2-8/8/12 |
14-18/8/12 |
3-7/9/12 |
Latest change |
Sample base |
1,000 |
1,018 |
1,002 |
1,019 |
1,005 |
-- |
Overall response rate |
69.1% |
65.8% |
65.9% |
66.6% |
64.6% |
-- |
Latest finding |
Finding |
Finding |
Finding |
Finding |
Finding |
-- |
Public Sentiment Index (PSI) |
92.3 |
84.7 |
86.0 |
79.7 |
75.7 |
-4.0 |
Government Appraisal (GA) |
94.8 |
89.0 |
90.0 |
85.1 |
82.0 |
-3.1 |
Society Appraisal (SA) |
95.2 |
89.9 |
90.8 |
86.4 |
83.5 |
-2.9 |
[17] Public Sentiment Index originates from the “now Survey on Public Sentiment Index" in 2011. It was then sponsored by “now news channel”.
Figures show that the latest Public Sentiment Index (PSI) is 75.7, down 4.0 when compared to last survey. The Government Appraisal (GA) and Society Appraisal (SA), the scores are 82.0 and 83.5, down 3.1 and 2.9 from last respectively.
|
|
Commentary
Note: The following commentary was written by Director of POP Robert Chung.
Our latest survey conducted before the Legislative Council Election shows that the popularity of CE CY Leung continues to drop, with support rating of 46.5 marks and approval rate of 33%. Both figures are at their record lows since he was elected as the CE. His net popularity now stands at negative 17 percentage points.
As for the Secretaries of Departments, the latest support rating of CS Carrie Lam is 58.1 marks, her approval rate is 55%, disapproval rate 14%, giving a net popularity of positive 41 percentage points. Latest support rating of FS John Tsang is 54.3 marks, approval rate 53%, disapproval rate 12%, giving a net approval rate of positive 41 percentage points. As for SJ Rimsky Yuen, his support rating is 49.1 marks, approval rate 24%, disapproval rate 6%, and his net popularity stands at positive 18 percentage points. Carrie Lam remains to be the most popular Secretary of Department.
As for the Directors of Bureaux, compared to one month ago, the net approval rate of 5 of the 12 Directors have gone up while 7 have gone down. Among them, Secretary for Environment Wong Kam-sing, Secretary for Security Lai Tung-kwok, Secretary for Labour and Welfare Matthew Cheung, Secretary for Financial Services and the Treasury Ceajer Chan and Secretary for the Civil Service Paul Tang have registered changes in net approval rates beyond sampling error, up by 10, 9 and down by 9, 7 and 7 percentage points respectively. Among all the Directors, only Secretary for Development Paul Chan and Secretary for Education Eddie Ng register negative popularity, at negative 29 and 32 percentage points respectively. Secretary for Food and Health Ko Wing-man continues to be the most popular Director.
According to POP’s standard, Ko Wing-man falls under the category of “ideal” performer, Matthew Cheung, Carrie Lam, Lai Tung-kwok and John Tsang now fall under the category of “successful” performer. The performance of Anthony Cheung, Tsang Tak-sing, Eddie Ng and Paul Chan can be labeled as “mediocre”, and that of Ceajer Chan, Raymond Tam, Wong Kam-sing, Gregory So, Rimsky Yuen and Paul Tang can be labeled as “inconspicuous”. CY Leung can be labeled as “depressing” while no one falls into the category of “disastrous”.
The following table summarizes the grading of CE and the principal officials for readers' easy reference:
"Ideal": those with approval rates of over 66%; ranked by their approval rates shown inside brackets |
Secretary for Food and Health Ko Wing-man (69%) |
|
"Successful": those with approval rates of over 50%; ranked by their approval rates shown inside brackets |
Secretary for Labour and Welfare Matthew Cheung Kin-chung (55%[18]); CS Carrie Lam Cheng Yuet-ngor (55%[18]); Secretary for Security Lai Tung-kwok (54%); FS John Tsang Chun-wah (53%) |
|
"Mediocre": those not belonging to other 5 types; ranked by their approval rates shown inside brackets |
Secretary for Transport and Housing Anthony Cheung Bing-leung (41%); Secretary for Home Affairs Tsang Tak-sing (34%); Secretary for Education Eddie Ng Hak-kim (17%); Secretary for Development Paul Chan Mo-po (16%) |
|
"Inconspicuous": those with recognition rates of less than 50%; ranked by their approval rates; the first figure inside bracket is approval rate while the second figure is recognition rate |
Secretary for Financial Services and the Treasury Ceajer Chan Ka-keung (38%, 45%); Secretary for Constitutional and Mainland Affairs Raymond Tam Chi-yuen (36%, 49%); Secretary for the Environment Wong Kam-sing (33%, 43%); Secretary for Commerce and Economic Development Gregory So Kam-leung (26%, 40%); SJ Rimsky Yuen Kwok-keung (24%, 30%); Secretary for the Civil Service Paul Tang Kwok-wai (20%, 29%) |
|
"Depressing": those with disapproval rates of over 50%; ranked by their disapproval rates shown inside brackets |
CE Leung Chun-ying (50%) |
|
"Disastrous": those with disapproval rates of over 66%; ranked by their disapproval rates shown inside brackets |
-- |
[18] In one decimal place, the approval rate of Secretary for Labour and Welfare Matthew Cheung was 55.0%, while that of CS Carrie Lam was 54.7%.
As for people’s public sentiment, our Public Sentiment Index (PSI) score continues to drop, both in terms of Government Appraisal (GA) and Society Appraisal (SA) scores, meaning that the public sentiment has turned worse.
|
Future Release (Tentative)
|
|
| Abstract | Latest Figures |Commentary | Future Release (Tentative) |
| Detailed Findings (Popularity of Chief Executive/ Popularity of Principal Officials/Public Sentiment Index Feature Page) |
|