• AFP – Vendor Performance Program: Q & A

    Q. Why are you introducing the IO AFP Vendor Performance Program?

    The AFP Vendor Performance Program is another example of IO’s commitment to continuous improvement. The program is an additional way to protect the public interest and hold construction contractors accountable their performance on projects.

    We want to encourage contractors to deliver the best possible results to our clients. This means using not only the robust tools within AFP project agreements, but also ensuring that future procurement processes are as rigorous as possible and take into account past performance in an efficient, fair and transparent manner.

    Q. What is IO’s AFP Vendor Performance Program?

    AFP Vendor Performance is IO’s program to monitor, track and inform construction contractors about their performance on AFP projects during the last 24 months.

    Construction contractors will be given a performance score, similar to a report card, whereby the contractor’s performance on AFP projects over the last 24 months will be assessed against specific AFP contract requirements.

    This construction contractor score, together with a report of performance infractions, will be updated on a monthly basis and disclosed to each construction contractor. This information will be commercially confidential and only provided to construction contractors.

    AFP Vendor Performance Program scores are based on disclosed, clear and measurable contract criteria. Construction contractors participating in the AFP program should not be surprised by their rating.

    The score will inform each construction contractor of the evaluation deductions, if any, that will be applied, if such construction contractor participates in an upcoming prequalification submission as part of a request for qualifications (RFQ) process.

    Q. What are the program’s main objectives?

    To ensure value for money and protection of the public interest, IO expects all contractors working on AFP projects for our clients to perform at their best.

    Contractors who perform well should continue to strive for the highest standards. Contractors who perform below their contractual requirements will want to consider how their past performance could be a factor in future procurements.

    Dialogue amongst private sector partners is important when forming partnerships and putting together teams to successfully bid on and build infrastructure projects. Bidders seeking to prequalify on AFP projects will want to consider the vendor performance score of construction contractors at the time of selecting team partners.

    Q. How will the AFP Vendor Performance Program score be created?

    As part of its due diligence in procurement and project management, IO has collected performance information on contractors for all AFP projects that have been in construction over the last 24 months. This performance information is based on a clear, precise and legitimate set of criteria that are monitored as part of IO’s regular role in contract management. The set of criteria are the basis for a score for construction contractors. All criteria are weighted the same.

    Each construction contractor’s score will be automatically calculated based on the number of times that a performance infraction (as described by the pre-established criteria) has occurred and been recorded in the last 24 months.

    The larger the number of performance infractions, the greater the deduction that will be applied against that construction contractor its next submission to an RFQ.

    Q. What criteria are used to determine infractions?

    The following list is a summary of the criteria that IO will use to monitor vendor performance. Each criterion is linked to a clear provision (or set of provisions) within the AFP project agreement. Detailed descriptions of these criteria, together with applicable interpretation notes, are disclosed in the sample contractor correspondence, available at here.

    1. The Contracting Authority has exercised remedial rights against the construction contractor (e.g. requiring corrective action) as a result of breach of the project agreement, which in the opinion of Infrastructure Ontario, is caused by the construction contractor.
    2. The Contracting Authority has sought indemnification in excess of $100,000 for losses suffered by the Project Sponsor / Contracting Authority due to the breach of the project agreement, which in the opinion of Infrastructure Ontario, is caused by the construction contractor.
    3. The Contracting Authority has set off proceeds in excess of $100,000 under the project agreement due to the breach of the project agreement, which in the opinion of Infrastructure Ontario, is caused by the construction contractor.
    4. The Contracting Authority has notified the project’s independent certifier and the independent certifier has agreed that construction work is defective and would cause the project not to achieve substantial completion.
    5. The Contracting Authority has uncovered defective work and/or has had to remedy the construction contractor’s defective work.
    6. A stop work order has been issued on the project which results in a follow-on notice of violation, fine or other disciplinary action against the construction contractor and/or the construction contractor has not implemented health and safety recommendations pursuant to an independent health and safety inspection report.
    7. The Contracting Authority has drawn upon withheld amounts in respect of the rectification of minor deficiencies.
    8. The Contracting Authority has drawn upon the contractor’s performance security.
    9. The construction contractor has failed to deliver a rectification schedule in response to the specific project agreement requirement to deliver such schedule where work is trending late.
    10. Substantial Completion, net of all resolved claims related to delay events under the project agreement, occurs more than 90 days after the Scheduled Substantial Completion Date.
    11. Substantial Completion, net of all resolved claims related to delay events under the project agreement, occurs after the Anticipated Substantial Completion Date.
    12. Pre-approved key individuals working for the construction contractor are substituted without the prior approval of the Contracting Authority (where such approval is required).

    Q. How does a record of infractions turn into a contractor’s score? How does scoring work?

    Recorded instances of infractions in the last 24 months are calibrated to a specified percentage point deduction from the “Construction Team Member Capability and Experience” section of the RFQ.

    The calibration of point deductions from recorded infractions is based on the objective of addressing performance in two ways:

    1. determining the threshold number of infractions that a construction contractor has accumulated, at any given time, before that construction contractor is statistically more likely of moving from the last qualifying position (e.g. 5th place in a Build-Finance or 3rd place in a Design-Build-Finance or Design-Build-Finance-Maintain RFQ) into a non-qualifying position (e.g. 6th place in a Build-Finance or 4th place in a Design-Build-Finance or Design-Build-Finance-Maintain RFQ); and
    2. determining the threshold number of infractions that a construction contractor has accumulated, at any given time, before that construction contractor is statistically more likely of failing to achieve the 60% minimum score threshold required of the Construction Team Member Capability and Experience.

    In order to determine the precise percentage of “Construction Team Member Capability and Experience” points to be deducted which would achieve the objectives stated above, a statistical model has been developed using the past eight years of RFQ scoring data for all IO AFP projects.

    The calibration of infractions into the percentage of “Construction Team Member Capability and Experience” will be disclosed to all bidders.

    Q. How much of an impact will a construction contractor’s score have on a team’s evaluation in an RFQ?

    The AFP Vendor Performance Program sends a signal to the marketplace that performance on projects matters. While it would require a contractor to have a series of significant infractions over a sustained period of time in order to seriously impact their standing in future procurements, construction contractors and their partners should not take any infractions lightly.

    For most contractors and most RFQs, deductions will not have a significant impact on the outcome of whether a team prequalifies or does not prequalify. IO has done statistical analysis of past projects and can provide the following insight on the consequence of contractor performance that needs improvement.

    In order for a construction contractor’s infraction record to cause it to move from a prequalifying position to a non-prequalifying position, a construction contractor must accumulate at least four infractions in the 24 month period before the RFQ submission deadline and must also rank at or very close to the 3rd place position of a DBF or DBFM RFQ, or at or very close to the 5th place position of a BF RFQ.

    In order for a construction contractor’s infraction record to cause it to move from a prequalifying position to an outright failure of the minimum scoring threshold of an RFQ (and cause its entire team to fail to prequalify), a construction contractor must accumulate at least eight infractions in the 24 month period before the RFQ submission deadline.

    Since the performance criteria are material in nature to any given project and represent significant performance problems, it would require repetitive material non-performance by any given contractor in order for such contractor (or its team) to be placed in jeopardy of not prequalifying on an AFP RFQ.

    Q. How were the thresholds for accumulations of infractions selected?

    The four infraction and eight infraction thresholds, which as described above are the points at which a construction contractor would statistically be more likely to move from the last-qualifying position into a non-qualifying position or into a failing position, respectively, were selected by IO taking into consideration the materiality of the infraction criteria and the observed frequency of such infractions in the history of AFP projects.

    It would require repetitive material non-performance by any given contractor in order for such contractor (or its team) to be placed in jeopardy of not prequalifying on an AFP RFQ.

    Q. Will the thresholds for the accumulations of infractions or the criteria be revisited?

    The program infraction criteria and thresholds for infraction accumulation as well as the criteria themselves will be reviewed within the first 18 months of program implementation and annually on a go forward basis.

    Q. It seems that some of the events described by the criteria are more significant or material than others. Why are the criteria weighted equally?

    The program infraction criteria have been developed to address a variety of performance issues that have been observed in past AFP projects. While some events may have different commercial, reputational, or programmatic risks, performance in all instances is important to IO. It is important to avoid the unintended consequences construction contractors not taking seriously certain infractions because of a lower deduction value.

    Q. How confident can I be that the impact of infraction records won’t cause unintended consequences?

    For most contractors and most RFQs, deductions will not have a significant impact on the outcome of whether a team prequalifies or does not prequalify.

    The point deduction calibrations, which have been developed using a data-driven statistical model, if applied to past IO RFQs, would not have had a significant impact in most instances.

    As of November 2016, out of the 43 AFP RFQs that closed and were evaluated since 2008 (9 BF, 13 DBF, 15 DBFM Social and 6 DBFM Civil):

    • 52 applicant teams (21.5% of total applicants) would have had a change in RFQ ranking, but only in 3 out of those 52 instances would the change in ranking have caused a change in the prequalified party shortlist; and
    • in only two instances (of 242 total applicants) would a team have failed to meet the minimum scoring threshold of 60% required for the Construction Team Member Capability and Experience section. Even in these two instances, the team that would have failed was already not in a prequalifying position.

    Q. How often will IO review the program to ensure it remains relevant and accurate? Will the scoring approach or infractions be subject to change?

    The program infraction criteria and thresholds for infraction accumulation will be reviewed within the first 18 months of program implementation and annually on a go forward basis.

    The scoring calibrations will be updated annually, taking into account the updating of the statistical model based on incoming RFQ scores for RFQs that have been procured during the year.

    Q. How are you introducing the program to contractors and when do contractors have to be ready for this to take effect?

    The application of construction contractor scores at the RFQ stage will commence for all AFP RFQs that are issued after January 1, 2017.

    In November and December 2016, construction contractors were contacted by IO and given a snapshot of their accumulated infractions to date (i.e. all infractions recorded on the IO AFP projects in construction for the last 24 months in which they have been the construction contractor) and a sample scorecard that reveals the RFQ point deduction (from the “Construction Team Member Capability and Experience” section of the RFQ) that would be applied against them if the program were fully implemented as of today.

    Q. How will the program be phased in?

    The program will be phased in in order to assist construction contractors in adjusting to the implementation of the AFP Vendor Performance Program. Specifically, the scores will be applied with a 50% weighting for RFQs that are issued between January 1 2017, to January 1, 2018. This means that only half of the RFQ point deductions in the calibration tables will be applied during the first year. Starting January 1, 2018, the full scoring deduction (if any) will be applied at the RFQ stage.

    The program infraction criteria and thresholds for infraction accumulation will be reviewed within the first 18 months of program implementation and annually on a go forward basis. Infrastructure Ontario is committed to the principle of continuous improvement and will use data gathered during the first phase of the program to assess the effectiveness of the program.

    Q. Why do potential deductions only apply during the RFQ stage and not as part of the RFP?

    From a legal and procurement perspective, once the assessment of criteria has occurred at an RFQ stage, for the purposes of qualifying bidders on their experience, the same criteria cannot be used again to assess the same bidder at the RFP stage. The RFQ measures general capability and experience, including past experiences. This is the appropriate stage to take past performance into consideration. By contrast, an RFP evaluation considers a bidder’s specific response to, and price for, a specific project.

    Q. How will vendor performance be administered and implemented?

    IO’s Procurement Department will administer the program.

    Administration of reporting during construction:

    The occurrence of any construction contractor infractions will be reported and reviewed each as part of Infrastructure Ontario’s regular internal monthly reporting of construction performance against the defined infraction criteria. Senior management at Infrastructure Ontario, including representatives of the project delivery, legal and procurement departments, will conduct due diligence to ensure that reporting is accurate and sufficient documentary detail exists to substantiate any given performance infractions.

    Infraction totals will then be updated by IO’s Procurement department and communicated to contractors via monthly scorecard letter correspondence. The scorecards are commercially confidential. Contractors, however, may determine for themselves whether to share their scorecards and with whom.

    Administration of applicable deductions during the RFQ process:

    A deduction to be applied against a construction contractor in any given RFQ will be the deduction set out for that construction contractor in the month in which the RFQ Prequalification Submission is received.

    For example, if as of January 1, 2017, a Contractor ABC’s point deduction for Design-Build-Finance RFQs is minus 1.75% (based on having accumulated two infractions in the last 24 months with a further 50% reduction during the 2017 phase in period), then for any prequalification submission received during the month of January 2017 which names Contractor ABC as the construction prime team member, 1.75% of the 35 points available for the Construction Team Member Capability and Experience in the RFQ will be automatically subtracted from that section following the completion of individual and consensus evaluations by scoring evaluators

    During the RFQ evaluation process, vendor performance point deductions will be applied to a bidding team’s score by the IO Procurement Department only after the scoring evaluators have reached consensus on their evaluation of prequalification submissions. This is a safeguard to ensure that there are proper checks and balances during evaluation and that the evaluation process continues to be fair. As with all evaluation processes, the application of vendor performance deductions will be overseen by third-party Fairness Monitors to the IO procurement process.

    Q. On which projects will these criteria be tracked and recorded?

    Infrastructure Ontario will track and record the stated performance criteria above for those project in which Infrastructure Ontario is defined as a co-Sponsor under the project procurement and for which Infrastructure Ontario has an official governance role during the construction phase of the project. Projects in which Infrastructure Ontario does not have a lead procurement or contract management role through to substantial completion will not apply to the dataset and no infraction criteria will be tracked for the purposes of the AFP Vendor Performance Program on those projects.

    Q. How will contractors be scored if they have not worked on a project in the last 24 months?

    Contractors who have not been involved in the construction of an AFP project for the last 24 months will not have a score until they accrue a score through participation in an IO AFP project.

    Q. How will contractors be scored if they have not been a part of an AFP project before?

    Contractors who have not been involved in the construction of an AFP project for the last 24 months will not have a score until they accrue a score through participation in an IO AFP project.

    Q. How will joint venture construction teams be assessed?

    IO recognizes that joint venturing is a successful and important team approach to undertaking AFP projects. From a performance perspective, joint ventures ought to share all risk and responsibilities among their constituent entities. This means that in monitoring performance, any instances of poor performance infractions will be attributed in full to all members of a joint venture.

    During the bidding phase, when firms are forming joint ventures, the number of infractions attributable to a bidding joint venture will be derived from the arithmetic average of the AFP Vendor Performance Program scores that each constituent member of the joint venture has at the time of the RFQ submission, based on the proportion of the joint venture interest.

    For example, if as of January 1, 2017, Contractor ABC’s point deduction for Design-Build-Finance RFQs is minus 1.75% (based on having accumulated two infractions in the last 24 months with a further 50% reduction during the 2017 phase in period) and Contractor XYZ does not have any record of infractions (and thus a 0% deduction), and Contractor ABC and XYZ are in a 50/50 joint venture bidding to an RFQ, then for any prequalification submission received up until February 28, 2017 in which Contractor ABC and Contractor XYZ are acting as 50/50 joint venture partners, the total deduction applied to the score of the joint venture will be minus 0.875% of the Construction Team Member Capability and Experience available points, which will be automatically subtracted from that section following the completion of individual and consensus evaluations.

    Q. What if one team’s scorecard changes after joint venture teams have formed? What is the other joint venture partner supposed to do?

    While contractors may form joint ventures many months in advance of an RFQ being issued, the AFP Vender Performance Program score of any given construction contractor is only taken into account for bidding purposes on the date of the submission of an RFQ. IO and its partners are not privy to the timing of formation of joint ventures, and the most fair and transparent means of applying a vendor performance score to a joint venture is at the time of RFQ submission.

    The respective members of a joint venture will continue to receive their own monthly updates of performance scorecards, and can use this information to determine their approach to an RFQ submission accordingly.

    Once an RFQ prequalification submission has been submitted, then it will not matter, for evaluation or scoring purposes in the RFQ, whether one or more of those contractors’ scores have changed since the date of RFQ submission. Any subsequent change in a contractor’s score will not have a further impact for that particular joint venture in that specific RFQ.

    Q. What can a construction contractor do when they do not agree with the poor performance infraction log? Can they appeal?

    The AFP Vendor Performance Program is intended to give a clear signal to the market that IO is serious about contractors performing well and that there are consequences for persistent and material poor performance. Contractors should not be surprised by any of the information they receive. A contractor’s future success is an incentive for sub-par performers to improve.

    All bidders are on a level playing field. The types of infractions described by the performance criteria above are very serious and will only occur in an AFP contract when a construction contractor has engaged in persistent and continuous sub-standard performance or behaviour. There will not be any ability to dispute or appeal whether an infraction described by any performance criterion has occurred.

    If there is an administrative error in the tracking of infractions we will correct our information. IO will ensure quality control of the data and processes that result in the scores.

    Q. How is due process ensured in respect of infraction events if there is no substantive dispute mechanism?

    The AFP Vendor Performance Program scoring adjustments relate to a construction contractor’s qualifications, experience and capability to perform work as assessed in an RFQ. Occurrences of persistent and continuous sub-standard performance or behaviour in recent AFP contracts, which is what the AFP Vendor Performance Program criteria measure, is extremely relevant for consideration at the RFQ stage.

    Consideration of past performance on projects in an RFQ (which is very heavily based on contractor’s use of past project examples to earn points) is very different from the determination of where legal liability is apportioned under a contract. The AFP Vendor Performance Program does not take away any of a contractor’s rights to dispute a specific event under a project agreement. Rather, the program is designed to capture the series of behaviours that lead to any given infraction being registered.

    For example, if the Contracting Authority has drawn upon the contractor’s performance security (e.g. a letter of credit), a performance infraction would be recorded against such contractor. That contractor may choose to dispute whether the Contracting Authority was legally entitled to draw upon the performance security in the first place – and may in fact win such legal challenge. Even in this situation, the AFP Vendor Performance Program would continue to register an infraction on the account that significant performance failures would have necessarily led up to the Contracting Authority needing to seek recourse on a project issue by drawing on performance security in the first instance. It is these types of performance failures which lead up to the occurrence of an infraction event which IO is seeking to discourage and eliminate through the implementation of the program.

    Q. You can already penalize a contractor through the project agreement. Isn’t it “double jeopardy” to be able to use that same issue as a scoring penalty for future bids?

    No, the AFP Vendor Performance Program infraction criteria capture not only the infraction events themselves, but the series of behaviours that lead to any given infraction being triggered. This is done without taking away any of a contractor’s rights to dispute matters under a project agreement.

    Project agreements address specific incidents based on the provisions of the agreement. The AFP Vendor Performance Program infraction criteria measure not only whether an event occurred, but capture the series of behaviours over a period of time that led up to such event.

    More importantly, the AFP Vendor Performance Program scoring adjustments relate to a construction contractor’s qualifications, experience and capability to perform work as assessed in an RFQ. Consideration of these occurrences of significant poor performance events in an RFQ is very different from the determination of where legal liability is apportioned under a contract.

    Q. How can a contractor improve its standing if it has a poor scorecard?

    The infraction criteria that define sub-standard performance are material in nature and will only be triggered when significant performance issues present themselves on a project. Nevertheless, infractions recorded for construction contractors will expire after 24 months from the date of the infraction being logged. This ensures that where infractions have occurred, they are not unfairly held against a construction contractor indefinitely. An improvement in performance within a given project will result in the benefits afforded under the Project Agreement itself.

    By continuing to perform in accordance with the terms of the Project Agreement, a contractor is assured that in time, all infractions will automatically fall away.

    Q. Is there a higher probability of getting infractions where projects are complex or where companies have done more work? Will consideration be given to adjustments in the infraction weighting or calibration for such companies?

    All projects are important to Infrastructure Ontario and it is our expectation that each contractor perform to its best in each project that it has been awarded. While all projects have their share of risks that must be managed by the contractor, all contractors have the opportunity to understand those risks during the procurement process. That being said, it is not necessarily true that more complex projects will result in a higher probability of infractions being recorded, since many of the criteria are directed to behaviours of poor performing contractors, not the risk of a project.   

    More importantly, there is no objective way of distinguishing what is a ‘complex’ project from other projects and there is no fair way of determining what threshold number of projects would even entitle any contractor to special accommodations. In short, trying to adjust infraction weightings or calibrations to take into account project risk or volume would be even more unfair to contractors bidding to AFP projects and would arbitrarily result in IO favouring certain bidders over others.

    Q. The AFP Vendor Performance Program seems to focus on criteria relating to performance failures. Why isn’t good performance rewarded? Can a contractor improve performance during the 24 month period and have prior infractions removed and/or retracted?

    Good performance is performance in accordance with the project agreement. The incentives to perform well within the project agreement are already clear.

    Performance infractions will only be tracked against a construction contractor for 24 months, which is a reasonable length of time considering that the construction of a typical AFP project may be significantly longer than two years. Infraction records will be updated on a rolling monthly basis.

    Q. Will contractor performance monitoring include assessment of ethical behaviour and conflicts of interest?

    The AFP Vendor Performance Program is focused solely on performance under AFP projects in construction and does not include any criteria relating to ethical bidding behaviour, conflict of interest disclosure, or other practices engaged during the bidding process.

    AFP procurement documents and IO procurement process already have significant safeguards to address ethical bidding practices, disclosure of perceived, potential, and actual conflicts of interest, and engagement by bidders or contracting parties in illegal acts.

    Q. Is introduction of the AFP Vendor Performance Program the only thing being done to improve contractor performance?

    AFP procurement and contract documents have already been recently enhanced to require higher standards that affect numerous contractor responsibilities, including for example:

    • increased demonstrated stakeholder and communications expertise;
    • additional rigour regarding construction scheduling; and
    • improved safety standards such as the Certificate of Recognition (or OHSAS 18001) standard and minimum WSIB CAD-7 scores among applicants in the prequalification stage.

    Q. Is this program a reaction to a number of AFP contractors’ recent performance?

    Our ‘on budget’ track record is second to none. Our ‘on time’ track record can always be improved. We want contractors to deliver results and maintain our high standard for project delivery. The success of the AFP Vendor Performance Program lies in the fact it will be easy to administer and be fair for all contractors.

    Q. Why is this program only being implemented for construction contractors and not subcontractors? Why should a general contractor or construction prime team member be penalized for a sub-contractor’s poor performance?

    The AFP RFQ does not pre-qualify subcontractors – only prime construction contractors. Selection of construction subcontractors and the ultimate oversight of subcontractors under the project agreement is, and always has been, the responsibility of the general or prime construction contractor.

    Q. Why is this program only being implemented in respect of construction contractors? Why not other team members of a consortium (e.g. equity providers, designers, facilities maintenance providers, etc.)

    Implementing our AFP Vendor Performance Program to monitor constructor-specific issues is the best first step toward addressing persistent sub-par performance and poor behaviour and represents a positive step in our commitment to continuous improvement.

    IO now has the benefit of 10 years of experience in its AFP program. The AFP Vendor Performance Program is commencing with a focus on construction contractors as opposed to other project components for several reasons:

    1. The construction phase of an AFP project is the stage that carries the greatest overall risks to the programs our public sector clients deliver. Construction delays and defective work directly impact the Government’s ability to deliver public services, whether in health care, justice, community safety, transit or transportation.
    2. Our experience has shown problems attributable to the construction phase and construction risks are the most frequent areas in which contractor poor performance is exhibited.
    3. The best and most objective data, metrics, and criteria we have regarding vendor past performance is in relation to the construction stage of projects and the role of construction contractors in delivering projects.

    Q. What is the difference between the AFP Vendor Performance Program and IO’s Real Estate Vendor Performance Program?

    A full description of IO’s Real Estate Vendor Performance Program is available here. Some of the differences between that program and the AFP Vendor Performance Program include:

    1. In AFP, the assessment of performance is based on a 24-month rolling calendar. In Real Estate, contract performance is reviewed by the PMSP provider and IO project managers at the conclusion of each assignment.
    2. In AFP, there are 12 very specific criteria which are monitored as part of ongoing monthly construction oversight. The assessment of whether criteria have been triggered is binary and does not require discretionary assessment by contract managers. In Real Estate, construction performance is based on a performance questionnaire which, though qualitatively objective, is not currently hard-coded to specific breaches of the contract. Thus, in Real Estate, scorecards are based on guidelines and expert judgement of contract management staff. Results of the questionnaire are then applied to a rating scale.
    3. In AFP, scoring will be updated monthly on a rolling basis and applied to projects that are ongoing and applied to the next AFP RFQ. In Real Estate, scoring is only updated at the end of an assignment (but a contractor’s rating does not expire) and ratings are applied in the next RFP.
    4. In AFP, other than to note obvious administrative errors, there is no dispute process to challenge the substantive assertion that an infraction has occurred. It has either happened or not and will have already been made known to the contractor through the standard contract management interactions it has on an active AFP project. In Real Estate, a rating can be disputed based on whether the judgment exercised by the scoring reviewers was fair and substantiated by facts recorded during the performance of the contract.
    5. In Real Estate, since there is so much volume of work coupled with a smaller scope, parity or evening out of scoring can occur amongst the entire vendor pool. Additionally, contractors who have not performed work for IO before are given the average performance rating of all other vendors. In AFP, assignments are fewer and larger in scope so it is easier to track historical data.

    Q. How do you define success? How will you know your program works?

    We simply want to improve the performance of contractors and reduce the instances of poor contractor performance.

    We want all contractors to meet the high standards we set as part of Ontario's infrastructure program. The AFP Vendor Performance Program will demonstrate its value when we reach the state when no contractors have any points deducted from their RFQ evaluation.

    Q. Doesn’t a clean slate inappropriately benefit new entrants into Ontario’s AFP market?

    No. All contractors must meet our stringent RFQ requirements and qualify on the merits of the RFQ itself. This includes scoring well in all categories, including health and safety, local knowledge, construction expertise, financial capacity, design and facilities management expertise (where applicable) and more. These important elements of the RFQ, which are largely represented through project examples, key individual resumes, and narrative approaches to organization and project delivery, continue to comprise the overwhelming majority of points in the RFQ.

    Contractors who have had experience in the AFP program will be familiar with IO’s procurement requirements and the key ingredients to achieve success, including required approaches for health and safety, construction scheduling, resourcing requirements, and partnering with IO. Contractors who have had success in the AFP program and performed well in the past should expect to continue to be successful in the future.

    Q. What will happen to contractors who consistently have infractions?

    Contractors who consistently fail to perform as specified will be forced to decide if they want to be a part of IO’s infrastructure program as their future participation could be jeopardized.

    Q. How many contractors will be assessed on a monthly basis? How many contractors participate in the AFP program in total and how many will be impacted by the AFP Vendor Performance Program?

    As of January 2017, there have been 15 construction contractors who have been involved in the construction of AFP projects over the last 24 months. These 15 construction contractors will receive monthly reports from IO. As new entrants to the AFP program enter into project agreements, they too will then be assessed on a monthly basis. Companies who have not been active in construction in the last 24 months or who have never performed construction with IO on an AFP project will not receive a monthly update and will not have a deduction applied.

    Q. What will a score card look like and who will get it?

    A sample scorecard/infraction report letter is available at here.

    Q. How is IO considering performance in procurements currently?

    Currently IO conducts a thorough assessment and evaluation of team member project experience, key individual experience, and the manner in which members of a team propose to work together and organize themselves to meet the project requirements. Other than the project examples that an applicant discloses and other rights that IO has to perform fact-checking or verification supplied, there has not been a mechanism to consider instances of past poor performance on IO AFP projects until now.