The Rural Health Care Program Initial Statistical Analysis of Data from the 2006/2007 Compliance Audits By Office of Inspector General Federal Communications Commission October 3, 2007 2 Background and Introduction This report contains a statistical analysis of disbursements analyzed in the 2006/2007 audits of the Rural Health Care (“RHC”) Program of the Federal Communications Commission (“FCC” or “Commission”). The audits examined funding provided during the Commission’s 2005 fiscal year and also included funds committed in Universal Service Fund (“USF”) funding years 2001-2004. The data suggest that additional oversight of the management of the RHC Program is needed. The primary objective of the Inspector General (“IG”) in auditing the RHC Program was to determine whether RHC payments by the Universal Service Administrative Company (“USAC”) complied with the FCC's rules and opinions interpreting those rules. In addition, the audits were intended to produce data that would permit statistical estimates of the erroneous payment rate and the amount of erroneous payments as defined in the Improper Payments Information Act of 2002 (“IPIA”).1 Under the IPIA, estimates of both the erroneous payment rate and the amount of erroneous payments may assist the Commission in assessing risks associated with the RHC Program. Under IPIA standards, a program is at risk if the erroneous payment rate exceeds 2.5 percent and the amount of erroneous payments is greater than $10 million. To assess compliance and risk, a simple random sample of auditees (i.e., funding requests of health care providers (HCP) approved by USAC for payment), was drawn and compliance attestation audits were completed. Statistical results from a random sample of 77 auditees suggest that the program is not “at risk” as defined by the IPIA, but there are significant problems in the program and an excessive rate of improper payments. The erroneous payment rate is 20.64 percent.2 Put somewhat differently, one in five funding requests approved by USAC for payment was not in compliance with the Commission's rules. Generally, there was compliance with Commission Rules and Regulations; however, of the 77 compliance attestation examinations/audits 10 audits (12.99 percent of the audits) are disclaimed opinions by auditors. In these ten audits, no opinion is provided as to whether the HCP or disbursement was in compliance with FCC Rules. The fundamental cause of the disclaimed opinions was insufficient information or documentation to permit the auditor to render an opinion on the HCP. When the FCC is unable to discern whether a payment was proper as a result of insufficient or lack of documentation, IPIA standards require that the payment be considered in error.3 1 Pub. L. 107-300, 116 Stat. 2350. 2 Statistical analysis of the 89 observation which includes non-random substitutes is set out in Appendix 1. 3 Memorandum For Heads of Executive Departments and Agencies – Issuance of Appendix C to OMB Circular A-123. Executive Office of the President, Office of Managemant and Budget (August 10 2006), Appendix C at 2. 3 Therefore, because the ten audits are disclaimed opinions, they are excluded from compliance calculations, but total disbursements of the disclaimed opinions/audits are included in the calculation of the erroneous payment rate. Description of Rural Health Care Program Telecommunications services and Internet access services can have an extremely positive impact on the delivery of health care services and patient well being, particularly in rural areas. The RHC Program makes discounts available to eligible rural HCPs for telecommunication and Internet services. In establishing the RHC Program, the Congress and the Commission sought to ensure that rural HCPs pay no more for telecommunications in the provision of health care services than their urban counterparts.4 The level of support depends on the location and the type of services chosen and is calculated individually for each HCP. In the RHC Program, the HCP does not receive payment for eligible telecommunications or Internet services. Instead, HCPs apply for discounts from prices of services provided by telecommunications and Internet service providers, and the RHC Program reimburses these service providers for telecommunications and Internet services provided to rural HCPs. The USAC administers the RHC Program of the USF.5 Overview of Administrative Process Generally, all HCPs or consortia of HCPs seeking to participate in the RHC Program must complete a Description of Services Requested and Certification Form (Form 465) to request bids from service providers. Form 465 must be completed for each physical location in order to receive support. When a Form 465 is received from a new applicant, USAC confirms eligibility and determines the completeness of the Form 465. Then, USAC posts the form on the USAC website,6 and a letter is sent to the HCP to confirm the posting. Posting invites service providers to bid to provide requested services, and the posting date starts the 28-day competitive bidding process.7 Any HCP expecting support must complete the 28-day posting requirement before entering into an agreement to purchase services with a service provider. After the posting period, a HCP must consider all bids received and must select the most cost-effective method to meet its communications requirements.8 In order to be eligible to receive telecommunications support, the HCP must select a "common carrier.” 4 See 47 U.S.C. § 254(h)(1) and (2). 5 See 47 C.F.R. §§ 54.701-705. See generally USAC Website: www.usac.org/rhc. 6 54 C.F.R. § 54.603(b). 7 54 C.F.R. § 54.603(a). 8 54 C.F.R. § 54.603(b)(4). 4 After services and service provider(s) are selected, the HCP completes and submits the Funding Request & Certification Form (Form 466) or an Internet Service Funding Request & Certification Form (Form 466-A) or both. Using these forms, the HCP specifies type(s) of service ordered, the cost of those services, the service provider(s), the terms of any service agreements, and certifies that the selections were the most cost-effective offers received. USAC reviews the Form 466 and/or Form 466-A packet for accuracy. After USAC’s approval, the HCP is mailed a Funding Commitment Letter and a copy of the Receipt for Service Confirmation Form (Form 467). A copy of the Funding Commitment Letter is also sent to the service provider. After the service is initiated by the service provider, the HCP submits Form 467 to USAC. Upon approval of the form, USAC sends the HCP and its service provider(s) a health care support schedule, upon receipt of which, the service provider can begin crediting the bill with the monthly recurring support amount or issue a check for the RHC discount.9 Once the service provider has issued a credit or check to the HCP, the service provider invoices USAC. USAC then credits or reimburse the carrier's USF account. For those providers that do not have an active USF account, FCC Form 498 must be filed, after which, upon USAC’s approval, reimbursement is issued by check or direct deposit.10 Compliance and IPIA Audits In early 2006, the IG established two objectives that each audit of the RHC Program was to achieve. Because of the comparatively small size of the RHC Program and of the determination11 that the RHC Program was not at risk under the IPIA, given the requirement that to be at risk required improper payments in excess of $10 million, the primary objective of the audits was to determine the extent to which HCPs were in compliance with FCC rules, orders and interprative opinions. Another objective was to provide the basis for a statistical measure of the erroneous payment rate so as to better inform future decision making under the IPIA. In order to determine compliance (as captured within the general administrative process described above), a compliance attestation audit of each service HCP on a specific FRN/WONUM12 was undertaken. With compliance attestation audits, the auditee (i.e., the management of the HCP) is required to sign an assertion letter acknowledging its responsibility for compliance with applicable requirements of Commission rules (e.g., 47 C.F.R. §§ 54.601-54.625), as well as applicable FCC orders, and to make specific assertions relative to the HCP’s compliance with those rules. Auditors validate or invalidate the assertions and provide 9 The issuance of a check for the discount means that the health care provider paid full price for the telecommunications or internet access and receives the discount in the form of a check. 10 USAC Website: www.usac.org. 11 See FCC Report to Congress on Improper Payments, March 31, 2004. 12 A FRN is the Funding Request Number assigned to the project for which USF support is requested. WONUM is the acronym for a work order number that is associated with a funding request. 5 the cause or causes for the failure of an assertion. That is, auditors determine whether the beneficiary of the RHC Program is in compliance with FCC rules, and if the beneficiary is not in compliance, the auditor identifies the cause(s) of, or reason(s) for, non- compliance. Table 1 contains the Assertion Letter that each auditee signed. Data generated from compliance attestation audits, which were based on the assertions set out in Table 1 below, were then analyzed statistically. TABLE 1 Example Assertions Letter Report of Management on Compliance with Applicable Requirements of 47 C.F.R. Subpart G of the Federal Communications Commission’s Rules, Regulations and Related Orders 13 Management of (name of Rural Health Care provider ) is responsible for ensuring that it is in compliance with applicable requirements of 47 C.F.R. §§ 54.601 through 54.625 of the Federal Communications Commission’s (FCC) Rules, Regulations and related FCC Orders for Rural Health Care Service Support. Management has performed an evaluation of its compliance with the applicable requirements of 47 C.F.R. §§ 54.601 through 54.625, as amended, with respect to receiving discounts on telecommunications and/or Internet Access services during the year ended September 30, 2005. The (Name of Rural Health Care provider) makes the following assertions that it is in compliance with applicable FCC Rules, Regulations and related FCC Orders (which are identified herein with each assertion) with respect to telecommunications and/or internet access discounts received, which were paid from the Universal Service Fund during the year ended September 30, 2005: A. Eligibility – the (name of Rural Health Care provider) asserts that: 1. it is a “health care provider” defined as: i. Post-secondary educational institution offering health care instruction, including a teaching hospital or medical school; ii. Community health center or health center providing health care to migrants; 13 Each assertion notes the applicable section of 47 C.F.R. Subpart G and the Subpart is provided as a separate enclosure. 6 iii. Local health department or agency; iv. Community mental health center; v. Not-for-profit hospital ; vi. Rural health clinic (including mobile clinics); or vii. Consortium of health care providers consisting of one or more of the entities described above (47 C.F.R. § 54.601(a)(2)); viii. Part-time eligible entities located in otherwise ineligible facilities; or ix. Emergency departments of rural for-profit hospitals (In the Matter of Rural Health Care Support Mechanism, Report and Order, Order on Reconsideration, and Further Notice of Proposed Rulemaking, Docket No. 02-60 18 FCC Rcd 24,546, FCC 03-288 ¶¶13, 15 (2003)); 2. as the entity receiving service, it is physically located in a rural area (47 C.F.R. §§ 54.601(c)(1),(2)); 3. certifies that the service(s) will be used solely for purposes reasonably related to the provision of health care services or instruction that the provider is legally authorized to provide under the law in the state in which such health care services or instruction are provided (47 C.F.R § 54.615(c)(4)). B. Services – the (name of Rural Health Care provider) asserts that: 1. it receives telecommunications and/or Internet access services based on a bona fide request (47 C.F.R. §§ 54.601(c)(1), 54.621); 2. the length of the supported service does not exceed the distance between the provider and the point farthest from the provider on the jurisdictional boundary of the largest city in the state (47 C.F.R. §§ 54.601(c)(1),(2), 54.625(a)); 3. Internet access and limited toll-free access to the Internet enables the (Name of Rural Health Care provider) to post its own data, interact with stored data, generate new data, or communicate over the World Wide Web (47 C.F.R. § 54.601(c)(2)); and 4. if it engages in eligible and ineligible activities or co-locates with an entity that provides ineligible services, it allocates eligible and ineligible activities in order to receive a prorated discount for eligible activities and chooses a method of cost allocation that is based on objective criteria and reasonably reflects the eligible use of the facilities (47 C.F.R. §54.601(d)). C. Competitive Bidding -- the (name of Rural Health Care provider) asserts that: 1. it submitted a properly completed FCC Form 465 to the Rural Health Care Division, signed by the person authorized to order telecommunication services 7 and includes the [following] required certifications under oath (47 C.F.R. § 54.603(b)); 2. the services were used solely for purposes reasonably related to the provision of health care services or instruction that the health care provider is legally authorized to provide under law in the state in which the services or instruction is provided and (a) the service(s) were not sold, resold or transferred in consideration of any other thing of value and (b) if the service(s) were being purchased as part of an aggregated purchase with other entities or individuals, the full details of any such arrangement, including the identities of all co-purchasers and the portion of the service(s) being purchased by the provider were disclosed (47 C.F.R. §§ 54.603(b)(iv) through (vi)); 3. it sought competitive bids for the service(s) and waited at least 28 days from the date on which its FCC Form 465 was posted on the USAC website, before making commitments with the selected telecommunications carrier(s) (47 C.F.R. § 54.603(b)(3)); 4. it selected the most cost-effective method of providing the requested service(s), where the most cost-effective method is defined as the method that costs the least after consideration of the features, quality of transmission, reliability and other factors deemed relevant to choosing the method of providing the service(s) (47 C.F.R. § 54.603(b)(4)); 5. it submitted paper copies to the USAC Administrator of the responses or bids received in response to the requested service(s) (47 C.F.R. § 54.603(b)(4)); and 6. if using a previously existing contract for service(s), the contract is exempt from the competitive bid requirements set forth in 47 C.F.R. § 54.603(a) consistent with the requirements of 47 C.F.R. § 54.604. D. Audits and Recordkeeping -- the (name of Rural Health Care provider) asserts that: 1. it maintained for its purchases of services supported under this subpart the same kind of procurement records that it maintained for other purchases (47 C.F.R. § 54.619(a)(1)(effective until March 16, 2004)), or, it maintained for its purchases of services supported, documentation for five years from the end of the funding year sufficient to establish compliance with all rules of 47 C.F.R. § 54.601 through 54.625 (47 C.F.R. § 54.619(a)(1)(as amended by Rural Health Care Support Mechanism, WC Docket 02-60, Report and Order, Order on Reconsideration, and Further Notice of Proposed Rulemaking, 18 FCC Rcd 24546, FCC 03-288, ¶ 49 (2003)); and 2. it will produce such records at the request of any auditor appointed by USAC 8 or any other state or federal agency with jurisdiction (47 C.F.R. § 54.619(b)). Under the IPIA, agencies are required to review all programs and activities they administer and to identify those that may be susceptible to significant erroneous payments.14 Significant erroneous payments are defined as annual erroneous payments in the program exceeding both 2.5 percent of program payments and $10 million,15 as noted above. While the RHC Program had been determined not to be a significant risk or have significant erroneous payments by USAC and by FCC management, the IG instructed USAC to provide a statistically valid estimate of the annual amount of improper payments in the RHC Program. When validating assertions or confirming compliance with Commission Rules, auditors were instructed to capture data on both over and under payments of disbursements, in order to estimate the error rate for erroneous payments and to provide an estimate of erroneous payments for IPIA purposes. Random Sampling In the efforts of the Office of Inspector General (“OIG”) to facilitate audits of Universal Service Programs/Activities for compliance with Commission rules and for compliance with the IPIA, the OIG provided a random sample for the RHC Program. Methodology: Sample Size and Variability The sample was designed to achieve a 95 percent confidence level that captures the proportion of providers that are: (1) not in compliance with Commission Rules, or (2) received incorrect payments. This sample design is based on the conclusions of USAC and the Commission’s Office of Managing Director that the RHC Program is in compliance with the IPIA and that bounding the proportion of RHC providers who are not in compliance is critical. To this end, sample size, n, was determined. Data were requested on a historical proportion,16 denoted p, of providers that were not in compliance with FCC rules and/or received incorrect payments. USAC provided several estimates, including 0.014, 0.04, and 0.083 (1 in 12). Because: (1) the range for the planning proportion varied, (2) the 0.014 number related to the percentage of erroneous payments, and (3) statistical practice requires np ? 5 and n(1 – p) ?5,17 an average planning proportion of 0.061 = (0.04 + 0.083)/2 was used. In addition, USAC provided 14 Memorandum For Heads of Executive Departments and Agencies – Issuance of Appendix C to OMB Circular A-123. Executive Office of the President, Office of Managemant and Budget. August 10 2006, p 3. 15 Ibid., p 4. 16 Expert judgment was provided as well as data. 17 David R. Anderson, Dennis J. Sweeny, and Thomas A. Williams. Essentials of Modern Business Statistics. Mason, Ohio: South-Western, 2004 at pp. 340-347. 9 for a margin of error, denoted E, of 0.05. In other words, the estimate must be within 0.05 of the true value in either direction. Using the standard statistical formula for sample size associated with proportions, the sample size is calculated as follows: 89)05.0( )061.01(061.0)96.1()1()( 2 2 2 2 2/ @-´´=-´´= E ppzn a The interval estimate for a Population Proportion is n ppzp )1( __ 2/ _ - ± a , where 2/az is the z value corresponding to an area of 2/a in the upper tail of the standard normal probability distribution. To calculate a confidence interval for a ratio of erroneous disbursements to total disbursements, the estimator åå== i i x y x yR _ _ ^ , where y and x are variables of interest. The confidence interval is )( ^ 2/ ^ RvzR a± , where the entire square root term is the estimated standard deviation of the ratio estimator.18 Sample Selection The sampling used here follows Anderson, Sweeny and Williams (2004). Given the sample size, n, and a complete listing of the universe that USAC provided, a simple random sample was selected of n WONUMs19 of HCP. To this end, random numbers were generated one for each record of the file which contained the population. The random numbers were generated with the function +RAND() of Microsoft Excel. Given that each record was assigned a unique random number, each unique FRN was actually assigned a unique random number. Thus, choosing n records corresponding to the n 18 All statistical formulas can be found in any standard general statistics textbook. For example, see William G. Cochran. Sampling Techniques. New York: John Wiley & Sons, Inc., 1963. 19 A WONUM (Work Order Number) in the Rural Health Care program is analogous to a Funding Request Number in the Schools and Libraries program. 10 smallest random numbers as in our sample is identical to choosing n WONUMs/FRNs (records) corresponding to the n smallest random numbers as our sample. 20 Substitution Errors Twelve observations from the RHC random sample could not be audited. Replacements were required. USAC selected twelve replacements to be audited; however none of the twelve replacement auditees was selected randomly. Moreover, none of the twelve replacements was taken from the list of extra random observations provided by the OIG for use when substitution became necessary. The use of the twelve non-random replacements in the sample of 89 observations would have created non- quantifiable substitution errors. Estimation of Erroneous Payment Rates and Compliance Rates As a consequence, our primary results are based on a random sample of 77 WONUMS (FRNS), where twelve non-random substitutes are deleted from the sample of 89 observations. The estimates from this random sample do not contain substitution errors. For this random sample of 77 observations, the estimated erroneous payment rate is 20.64 percent. The lower limit of a 90 percent confidence interval is 1.94 percent, and the upper limit of this 90 percent confidence the interval is 39.35 percent. The magnitude of the erroneous payment rate, however, suggests significant problems in the administration of the program.21 Non-compliance was found on assertions B1, B2, B4, C3, C4, C5, C6, D1, and D2 of Table 1. For the remaining assertions of Table 1, there was 100 percent compliance. All results from the random sample of 77 audits of the RHC Program are contained in Tables 2 and 3, below. TABLE 2 Erroneous Payment Rate Estimated Variance Estimated STD Margin of Error 90 percent Lower Limit 90 percent Upper Limit .2064198 0.01300768 0.114051 0.187044 0.019376 0.393464 20 David R. Anderson, Dennis J. Sweeny, and Thomas A. Williams. Essentials of Modern Business Statistics. Mason, Ohio: South-Western, 2004 at pp. 276-277. 21 Post-audit total disbursements in the RHC universe were $21,562,585.88 during the funding years 2001 through 2004. 11 TABLE 3 Exact Confidence Interval: Binomial Variable*** Random Sample Size = 77 31-Aug-07 Assertion A1 Confidence Confidence (THE AUDITEE IS A HEALTH CARE PROVIDER) Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 67 67 100.00% Assertion A2 Confidence Confidence (THE AUDITEE IS IN A RURAL AREA) Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 67 67 100.00% Assertion A3 (THE AUDITEE USES THE SERVICES FOR HEALTH CARE/INSTRUCTION) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 67 67 100.00% Assertion B1 Confidence Confidence (THE AUDITEE HAS MADE A BONA FIDE REQUEST) Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 64 67 5.00% 95.52% 87.47% 99.07% Non-Compliance* 3 67 5.00% 4.48% 0.93% 12.53% 1-Material Non-Compliance** 2 67 5.00% 2.99% 0.36% 10.37% Assertion B2 (DISTANCE CRITERIA SATISFIED) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 66 67 5.00% 98.51% 91.96% 99.96% Non-Compliance* 1 67 5.00% 1.49% 0.04% 8.04% 1-Material Non-Compliance** 12 Assertion B3 (SERVICES ENABLE INTERACTION) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 65 65 5.00% 100.00% Non-Compliance* 1-Material Non-Compliance** Assertion B4 (COSTS ALLOCATED AMONG CO-LOCATED ENTITIES) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 51 53 5.00% 96.23% 87.02% 99.54% Non-Compliance* 2 53 5.00% 3.77% 0.46% 12.98% 1-Material Non-Compliance** 1 53 5.00% 1.89% 0.05% 10.07% Assertion C1 (AUTHORITY TO SUBMIT 465) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 67 67 5.00% 100.00% Non-Compliance* 1-Material Non-Compliance** Assertion C2 (SERVICES SOLELY USED FOR HEALTH CARE/ INSTRUCTION) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 67 67 5.00% 100.00% Non-Compliance* 1-Material Non-Compliance** Assertion C2A (SERVICES NOT RESOLD OR TRANSERRED) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 67 67 5.00% 100.00% 13 Non-Compliance* 1-Material Non-Compliance** Assertion C2B (COSTS PROPERLY ALLOCATED) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 53 53 5.00% 100.00% Non-Compliance* 1-Material Non-Compliance** Assertion C3 (COMPETITIVE BIDS SOUGHT AT LEAST 28 DAYS PRIOR TO COMMITMENT) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 66 67 5.00% 98.51% 91.96% 99.96% Non-Compliance* 1 67 5.00% 1.49% 0.04% 8.04% 1-Material Non-Compliance** Assertion C4 (MOST COST EFFECTIVE METHOD) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 65 67 5.00% 97.01% 89.63% 99.64% Non-Compliance* 2 67 5.00% 2.99% 0.36% 10.37% 1-Material Non-Compliance** 2 67 5.00% 2.99% 0.36% 10.37% Assertion C5 (BIDS PROVIDED TO USAC) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 59 60 5.00% 98.33% 91.06% 99.96% Non-Compliance* 1 60 5.00% 1.67% 0.04% 8.94% 1-Material Non-Compliance** 1 60 5.00% 1.67% 0.04% 8.94% Assertion C6 (PRE-EXISTING CONTRACT, CONSISTENT WITH 47 C.F.R. §§ 54.603/4) Confidence Confidence Observed Sample Limit Limit 14 Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 60 61 5.00% 98.36% 91.20% 99.96% Non-Compliance* 1 61 5.00% 1.64% 0.04% 8.80% 1-Material Non-Compliance** Assertion D1 (RECORDS PROPERLY RETAINED) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 60 67 5.00% 89.55% 79.65% 95.70% Non-Compliance* 7 67 5.00% 10.45% 4.30% 20.35% 1-Material Non-Compliance** 4 67 5.00% 5.97% 1.65% 14.59% Assertion D2 (RECORDS PRODUCED ON REQUEST) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 59 67 5.00% 88.06% 77.82% 94.70% Non-Compliance* 8 67 5.00% 11.94% 5.30% 22.18% 1-Material Non-Compliance** 4 67 5.00% 5.97% 1.65% 14.59% * Non-Compliance includes both Material Non-Compliance and Non-Material Non-Compliance ** Assumes Material Non-Compilance =1 and Otherwise = 0 Otherwise includes both Compliance and Non-Material Non-compliance ***We use inverse beta which is the inverse of the incomplete beta function that approximates the binomial. Also see CRC Handbook of Tables for Probability and Statistics 2d ed. 1968. p. 219. 15 Causes of Non-Compliance When there was non-compliance on any assertion, data were collected on causes of non-compliance. Table 4 contains all 21 possible causes of non-compliance. Data were collected such that, if an auditor found multiple causes of non-compliance, all information would be presented. The statistical analysis of cause is contained in Table 5. TABLE 4 1-Imprecise FCC Rule/s 2-Contradictory FCC Rule/s 3-Overly Complex FCC Rules 4-Disregarded FCC Rule/s 5-Followed State Rule/s (apparent conflict with FCC Rule/s) 6-Followed USAC Procedures (apparent conflict with FCC Rule/s) 7-Inadequate Documentation Retention 8-Inadequate Auditee Processes and/or Polices and Procedures 9-Inadequate Systems for Collecting, Reporting, and/or Monitoring Data 10-Insufficient Resources/Time to Complete Task/Activity 11-Failure to Review/Monitor Work, Material, or Data/Application Submitted by Consultant/Agent 12-Applicant/Auditee Weak Internal Controls 13-Applicant/Auditee Data Entry Error 14-Service Provider Weak Internal Controls 15-Service Provider Data Entry Error 16-Service Provider Error (other) 17-USAC Error 18-SOLIX Error 19-NECA Error 20-Force Majeure (Acts of God and Nature) 21-Other 16 TABLE 5 Causality Exact Confidence Interval: Binomial Variable**** Random Sample Size = 77 Assertion B1 (BONA FIDE REQUEST) Confidence Confidence Non-Compliance: Causes* Observed Sample Limit Limit Occurences Size Significance Proportion Lower Upper X N Level P P_L P_U B1_8 (FAULTY AUDITEE PROCESSES/POLICIES) 2 3 5.00% 66.67% 9.43% 99.16% B1_11(WORK NOT MONITORED) 1 3 5.00% 33.33% 0.84% 90.57% B1_12 (WEAK AUDITEE INTERNAL CONTROLS) 1 3 5.00% 33.33% 0.84% 90.57% B1_16 (SERVICE PROVIDER DATA ERROR) 1 3 5.00% 33.33% 0.84% 90.57% Assertion B2 (DISTANCE CRITERIA ESTABLISHED) Confidence Confidence Non-Compliance: Causes* Observed Sample Limit Limit Occurences Size Significance Proportion Lower Upper X N Level P P_L P_U B2_8 (FAULTY AUDITEE PROCESSES/POLICIES) 1 1 5.00% 100.00% B2_16 (SERVICE PROVIDER DATA ERROR) 1 1 5.00% 100.00% Assertion B4 (COSTS ALLOCATED ELIGIBLE/INELIGIBLE) Confidence Confidence Non-Compliance: Causes* Observed Sample Limit Limit Occurences Size Significance Proportion Lower Upper X N Level P P_L P_U B4_8 2 2 5.00% 100.00% 15.81% (DATA ERROR) [ANTHONY CONFIRM] Assertion C3 (COMPETITIVE BIDS SOUGHT, WAITED 28 DAYS) Confidence Confidence Non-Compliance: Causes* Observed Sample Limit Limit Occurences Size Significance Proportion Lower Upper X N Level P P_L P_U C3_13 1 1 5.00% 100.00% (AUDITEE DATA ERROR) Assertion C4 (MOST COST EFFECTIVE Confidence Confidence 17 SOLUTION) Non-Compliance: Causes* Observed Sample Limit Limit Occurences Size Significance Proportion Lower Upper X N Level P P_L P_U C4_7 (AUDITEE DOCUMENT RETENTION FAILURE, COULD NOT PROVE) 1 2 5.00% 50.00% 1.26% 98.74% C4_11 (WORK NOT MONITORED, COULD NOT PROVE) 1 2 5.00% 50.00% 1.26% 98.74% C4_12 (WEAK AUDITEE INTERNAL CONTROLS, COULD NOT PROVE) 1 2 5.00% 50.00% 1.26% 98.74% Assertion C5 (BIDS PROVIDED TO USAC) Confidence Confidence Non-Compliance: Causes* Observed Sample Limit Limit Occurences Size Significance Proportion Lower Upper X N Level P P_L P_U C5_11 (WORK NOT MONITORED) 1 1 5.00% 100.00% C5_12 (WEAK AUDITEE INTERNAL CONTROLS) 1 1 5.00% 100.00% Assertion C6 (PRE-EXISTING CONTRACT, COMPLIES WITH 54.603/4) Confidence Confidence Non-Compliance: Causes* Observed Sample Limit Limit Occurences Size Significance Proportion Lower Upper X N Level P P_L P_U C6_7 (AUDITEE DOCUMENT RETENTION FAILURE) 1 1 5.00% 100.00% Assertion D1 (AUDITEE DOCUMENT RETENTION FAILURE) Confidence Confidence Non-Compliance: Causes* Observed Sample Limit Limit Occurences Size Significance Proportion Lower Upper X N Level P P_L P_U D1_7 (AUDITEE FAILED TO PROVIDE REQUIRED DOCS) 5 7 5.00% 71.43% 29.04% 96.33% D1_8 (INADEQUATE AUDITEE PROCESSES/POLICIES) 7 7 5.00% 100.00% D1_9 (SYSTEMS FAILURE) 5 7 5.00% 71.43% 29.04% 96.33% Assertion D2 (DOCUMENTS PRODUCED) Confidence Confidence Non-Compliance: Causes* Observed Sample Limit Limit Occurences Size Significance Proportion Lower Upper 18 X N Level P P_L P_U D2_7 (AUDITEE COULD NOT LOCATE/PRODUCE REQUIRED DOCUMENT) 6 8 5.00% 75.00% 34.91% 96.81% D2_8 (INADEQUATE AUDITEE PROCESSES/PLOICIES) 7 8 5.00% 87.50% 47.35% 99.68% D2_9 (SYSTEMS FAILURE) 5 8 5.00% 62.50% 24.49% 91.48% * Non-Compliance includes both Material Non-Compliance and Non-Material Non-Compliance **The assertion number followed by the underscore and number indicates the cause of non-compliance. For example D1_7 means non-compliance on Assertion D1 and a cause was 7 which is Inadequate Documentation Retention. ***On any assertion with non-compliance, the precentages associated with causes do not sum to 100 because data were collected such that mulltiple causes could be found and entered by an auditor. ****We use inverse beta which is the inverse of the incomplete beta function that approximates the binomial. Analysis/Conclusion Beneficiaries of the RHC Program are from several categories of HCPs. Community health centers or health centers providing care to migrants are 19.4030 percent of beneficiaries, and local health departments/agencies are 10.4478 percent of beneficiaries. Community mental health centers are 2.9851 percent of RHC Program beneficiaries, and not-for-profit hospitals are 49.2537 percent of beneficiaries. Finally, rural health clinics (including mobile clinics) are 17.9104 percent of RHC Program beneficiaries/funding requests. There was general compliance with FCC Rules and Regulations. Why are both the erroneous payment rate high and compliance rates high? Of the 77 compliance attestation examinations/audits 10 audits, or 12.99 percent of the audits, are disclaimed opinions by auditors. In these ten audits, no opinion is provided on whether the HCP is in compliance with FCC Rules. The fundamental cause of the disclaimed opinions was insufficient information/documentation to render an opinion on the HCP. When the FCC is unable to discern whether a payment was proper as a result of insufficient information or lack of documentation, IPIA standards require that the payment be considered in error.22 Therefore, because 12.99 percent of the audits are disclaimed opinions, the disclaimed opinions/audits are excluded from compliance calculations, but total disbursements of the disclaimed opinions/audits are included in the calculation of the erroneous payment rate and the erroneous payment amount. This explains the high compliance rates and the high erroneous payment rates. In summary, given the large number of disclaimed audits and the lack of appropriate documentation in the sample 22 Memorandum For Heads of Executive Departments and Agencies – Issuance of Appendix C to OMB Circular A- 123. Executive Office of the President, Office of Management and Budget. August 10 2006, p 2. 19 study, non-compliance may be more widespread than the results suggest. Our results suggest that for 12.99 percent of the universe of RHC providers/FRNs opinions cannot be rendered on compliance with FCC Rules and Regulations. Where auditors could render opinions on compliance/non-compliance with FCC rules observed causes of non-compliance are: Inadequate Documentation Retention; Inadequate Auditee Processes and/or Polices and Procedures; Inadequate Systems for Collecting, Reporting, and/or Monitoring Data; Failure to Review/Monitor Work, Material, or Data/Application Submitted by Consultant/Agent; Applicant/Auditee Weak Internal Controls; Applicant/Auditee Data Entry Error; and Service Provider Error (other). For both assertions dealing with the lack of required documentation, inadequate auditee processes and/or polices and procedures are present in10.45 percent of the population. Under the IPIA, estimates of both the erroneous payment rate and amount of erroneous payment are intended to guide the Commission in assessing risk that is associated with the RHC Program. To assess compliance and risk, a simple random sample of auditees, (in this case, funding requests of HCPs), was drawn and compliance attestation examinations/audits were completed. The statistical results from a simple random sample of 77 auditees suggest that the program is not at IPIA risk, but, with an estimated erroneous payment rate of 20.64 percent, there are significant problems in the program. 20 APPENDIX 1 This appendix contains the statistical analysis of the sample of 89 audits which includes 12 non-random substitutes. The estimation results are contained in TABLE A-1 and TABLE A- 2. Again, the estimates of this analysis contain substitution errors as a result of USAC’s inappropriate substitutions during the audit process. All substitution errors are non-quantifiable. TABLE A-1 Erroneous Payment Rate Estimated Variance Estimated STD Margin of Error 90 percent Lower Limit 90 percent Upper Limit 0.185677 0.010006971 0.100035 0.164057 0.02162 0.349735 The estimated erroneous payment rate under this incorrectly constructed sample is 18.57 percent. Compliance Results are presented in TABLE A-2. Again, disclaimed opinions are excluded from the calculations in TABLE A-2, because insufficient information or data did not permit auditors to complete ten compliance attestation examinations, i.e., auditors could not provide opinions on the compliance of ten HCPs’ compliance with FCC Rules and Regulations. TABLE A-2 Exact Confidence Interval: Binomial Variable*** Assertion A1 (HEALTH CARE PROVIDER) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 79 79 100.00% Assertion A2 (IN RURAL AREA) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 79 79 100.00% 21 Assertion A3 (HEALTH CARE/INSTRUCTION) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 79 79 100.00% Assertion B1 (BONA FIDE REQUEST) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 76 79 5.00% 96.20% 89.30% 99.21% Non-Compliance* 3 79 5.00% 3.80% 0.79% 10.70% 1-Material Non-Compliance** 2 79 5.00% 2.53% 0.31% 8.85% Assertion B2 (DISTANCE CRITERIA) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 78 79 5.00% 98.73% 93.15% 99.97% Non-Compliance* 1 79 5.00% 1.27% 0.03% 6.85% 1-Material Non-Compliance** Assertion B3 (SERVICES ENABLING) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 77 77 5.00% 100.00% Non-Compliance* 1-Material Non-Compliance** Assertion B4 (COSTS PROPERLY ALLOCATED) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 63 65 5.00% 96.92% 89.32% 99.63% Non-Compliance* 2 65 5.00% 3.08% 0.37% 10.68% 1-Material Non-Compliance** 1 65 5.00% 1.54% 0.04% 8.28% Assertion C1 Confidence Confidence 22 (AUTHORITY) Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 79 79 5.00% 100.00% Non-Compliance* 1-Material Non-Compliance** Assertion C2 (SOLELY FOR HEALTH CARE/INSTRUCTION) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 79 79 5.00% 100.00% Non-Compliance* 1-Material Non-Compliance** Assertion C2A (NOT RESOLD OR TRANSFERRED) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 79 79 5.00% 100.00% Non-Compliance* 1-Material Non-Compliance** Assertion C2B (COSTS PROPERLY ALLOCATED) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 65 65 5.00% 100.00% Non-Compliance* 1-Material Non-Compliance** Assertion C3 (COMPETITIVE BIDS SOUGHT WAITED 28 DAYS) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 78 79 5.00% 98.73% 93.15% 99.97% Non-Compliance* 1 79 5.00% 1.27% 0.03% 6.85% 1-Material Non-Compliance** 23 Assertion C4 (MOST COST EFFECTIVE) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 77 79 5.00% 97.47% 91.15% 99.69% Non-Compliance* 2 79 5.00% 2.53% 0.31% 8.85% 1-Material Non-Compliance** 2 79 5.00% 2.53% 0.31% 8.85% Assertion C5 (BIDS TO USAC) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 71 72 5.00% 98.61% 92.50% 99.96% Non-Compliance* 1 72 5.00% 1.39% 0.04% 7.50% 1-Material Non-Compliance** 1 72 5.00% 1.39% 0.04% 7.50% Assertion C6 (PRE-EXISTING CONTRACT) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 72 73 5.00% 98.63% 92.60% 99.97% Non-Compliance* 1 73 5.00% 1.37% 0.03% 7.40% 1-Material Non-Compliance** Assertion D1 (DOCUMENTS MAINTAINED) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 72 79 5.00% 91.14% 82.59% 96.36% Non-Compliance* 7 79 5.00% 8.86% 3.64% 17.41% 1-Material Non-Compliance** 4 79 5.00% 5.06% 1.40% 12.46% Assertion D2 (DCUMENTS PRODUCED) Confidence Confidence Observed Sample Limit Limit Occurrences Size Significance Proportion Lower Upper X N Level P P_L P_U 0-Compliance 71 79 5.00% 89.87% 81.02% 95.53% Non-Compliance* 8 79 5.00% 10.13% 4.47% 18.98% 1-Material Non-Compliance** 4 79 5.00% 5.06% 1.40% 12.46% 24 * Non-Compliance includes both Material Non-Compliance and Non-Material Non-Compliance ** Assumes Material Non-Compilance =1 and Otherwise = 0 Otherwise includes both Compliance and Non-Material Non-compliance. ***We use inverse beta which is the inverse of the incomplete beta function that approximates the binomial.