QPC Security - Breakfast Bytes

CISO, CTO, CIO, what’s the difference?

Kyle Wentworth of Wentworth Consulting Group joined Felicia to compare/contrast three C-suite level IT/IS related roles.
Kyle has 35 years of business experience and has been working on computers since 1976. He is a:
Fractional CTO
Business coach
Business process modeler
Kyle has a great resource on his website to help people understand the differences between these C-suite roles.
https://wentworthconsultinggroup.com/cto-cio-ciso-consulting/
Listen to the podcast for some Kyle truth bombs such as:
"Technology runs your business. You don't. We facilitate technology to run our business. IT is the most critical function of your business."
"Technology is harder to manage than brain surgery. Then why are you having conversations with technologists? Technology footprint is harder to identify and manage than brain surgery."
We talked about Felicia's hot button which is a lack of a quality, enforced procurement policy.
"The reason you give someone the ability to purchase the product is because you don’t understanding why they should NOT buy it."
 

Zero trust fundamentals

Zero trust is not a product you buy.The problem that most organizations have is that they are still not doing the fundamentals well.CIS has a community defense model.I did a detailed webinar on it where I covered a lot of these fundamentals.https://www.qpcsecurity.com/2023/02/16/addressing-information-security-fundamentals-with-cis-and-community-defense-model/
Let's look at inventory management, asset management, change management, onboarding and offboarding.
You must have checks and balances. There must be practices codified in policy with a shared responsibility model which make it so that the issues that are created by mistakes in onboarding or offboarding are caught.
Fundamentally, the most effective thing in zero trust are the protections that are in an always on state.Like for example the recent revelation about flaws in UEFI and SecureBoot.These have prerequisites like TPM, BIOS configs, bios adm pwds, automated firmware updates, procurement policy alignment for supported hardware, onboarding configuration done properly on those endpoints, monitoring of the firmware updates, and of course, no admin access for end users!!!
FUNDAMENTALS MUST BE MASTERED
When an organization does not have a CISO that has policy and management authority over IT, you are guaranteed to have problems.Forget CIO and CTO. I think those are old modes of thinking. Find a CISO that can be the leader of all IT strategy.
Procurement policy must include vetting and testing of cloud app integrations. Monitoring and technical controls must be in place to restrict or eliminate the ability of an end user to buy shadow IT and authorize it on their own. Azure AD has controls for this, but they are not on by default.

FTC SafeguardsRule, IRS requirements, and tax preparers

The IRS regulations for tax preparers being compliant with the FTC Safeguards rule is specified to be enforced starting in June 2023. It is doubtful that the majority of tax preparers are adequately compliant.
The IRS published information about this compliance requirement as far back as 2019.
https://www.irs.gov/newsroom/heres-what-tax-professionals-should-know-about-creating-a-data-security-plan
All of it is common sense and things that orgs should have been doing for ages.
IRS publication 4557
https://www.irs.gov/pub/irs-pdf/p4557.pdf
Before you use a tax preparer, ask them for their compliance certification statement.
Practical examples of what to ask of your tax preparer and why
https://qpcsecurity.podbean.com/e/business-survival-over-the-next-decade/
More information on this topic from Joe Brunsman, cybersecurity insurance expert.
https://youtu.be/NOY249doJXg
 

Methods to prevent business email compromise

Business survival over the next decade

What is the number one thing you can do as a consumer to protect yourself when dealing with tax preparers?
Practical examples of what to ask for from your tax preparer and why.
What are the total number of people that would have access to my records if I do business with you? You want me to sign a contract with you, terms and conditions that I have to abide by. If you are going to prepare my taxes, show me your affirmation statement where you as a tax prep preparer have put it in writing that you are fully in compliance as a business with the IRS requirements for tax preparers. Put that in writing.
If the IRS is the authority that is providing the designation that an organization is an IRS authorized tax preparer, then the IRS is the entity who defines the standard for what is the requirement put upon that organization or that person in order to have that designation. Therefore, it is completely legitimate to be asking as a prospective customer of that organization, "show me your compliance statements". How do you comply with the IRS requirements for tax preparers? And if you get anything other than a fully prepared premade statement they provided to you in writing,  then that's problematic because it means that they are not compliant.
What is one of the most important things that a business owner can do in order to make their business survive the next decade?
Information security risk management is everyone's problem.
Business leaders cannot delegate and abdicate involvement. 
If you are not having regular meetings with your vCISO, how can you make informed risk decisions? Do you know what the gaps backlog is for your organization? Do you have a risk register? If you refuse to make the time to meet regularly with your vCISO, your business is going to be squeezed by cybersecurity insurance requirements, governmental regulations, and customer requirements.
 
The executive management team needs to understand that if they do not tell all of the managers in an organization that they need to take responsibility for the ownership over their resources, then what needs to happen is that the executive management team needs to make the CISO or the IT department have full total authoritarian control over those resources. But then that turns into a big can of shut the heck up to the people who've abdicated their responsibility to be involved in the process. Because you can't have it both ways. You can't say that IT is responsible for the security of those assets, but then refuse to be involved in the conversations about who should be having access to what and when. And claim that you don't have time to talk about it, that it is not important. Of course it's important. Are you the resource owner or not? So you can't make it somebody else's responsibility to define the policy around who has access to that resource that ultimately you're responsible for and then yet get grumpy. when your access or the people who you thought should have had access to that resource have their access denied because IT is trying to clean up the mess. You can't have it both ways.
Whose responsibility is information security risk management? Ultimately, it's the executive management team. But they can delegate that through the organization to the resource owners and at the end of the day, IS risk management really needs to be everybody in the entire organization's responsibility. Information security practices need to permeate throughout the entire organization. The end users of an organization are the largest attack surface that an organization has.
Suggestions for tax preparers
Tax preparers need to comply with the FTC Safeguard rule which is currently slated to be enforced starting in June 2023. As of May 2023, the expected plan is that private contractors will be the enforcement auditing arm for compliance. 
In reality, any company that had taken cybersecurity insurance compliance preparedness and had engaged a vCISO proactively several years prior would likely have no issue in this area. But the vast majority of tax preparers were unwilling to invest in the kind of protections that should have been in place for decades. 
Here are some resources.
https://www.irs.gov/newsroom/heres-what-tax-professionals-should-know-about-creating-a-data-security-plan
https://www.irs.gov/pub/irs-pdf/p5293.pdf
https://nvlpubs.nist.gov/nistpubs/ir/2016/NIST.IR.7621r1.pdf
Page 13 of publication 4557 states that all tax preparers must comply with the FTC Safeguards rule. That means if you or your organization has an IRS tax preparer ID number, you must be in compliance and be able to prove that you are in compliance. 
Tax preparers that are under $2mm in revenue should expect to spend 15% of revenue annually on all inclusive IT costs. If your spend is not that high, then your organization is likely not going to be competitive in the market and is bound to lose market share to players who have invested in becoming FTC Safeguard rule compliant.
Please also be aware that security theater is not compliance. I have seen some scams such as do-it-yourself kits through technical firms who specialize in servicing accountants (per their website). 
https://www.irs.gov/pub/irs-pdf/p4557.pdf
 
More details from Joe Brunsman, cybersecurity insurance expert.
https://youtu.be/NOY249doJXg
 
 

PSA or ERP - paradigm and requirements analysis

I get a lot of questions about PSAs, ERPs, and overall paradigms related to core business software. This podcast summarizes things you should be thinking about in your software selection process.
After three years of investigating PSA and ERP options including spending a lot of money on software and payroll, the product we like is Odoo. Organizations using a PSA with add-ons approach are really missing the mark. There is no PSA that does project management well. None of them have accounting systems. Most of them are terrible at quoting. And they are all expensive. They also are all weak at analytics and business visualization or analysis.
So a company ends up paying for:
PSA
Quotewerks
Zomentum
eCommerce processor
Payment gateway provider
project management platform
QuickBooks or Xero
PowerBI
website hosting
Applicant tracking system
HR / people management system
email newsletter system
marketing automation platform
CRM
Social media marketing platform
and more
Whereas, a business could just get Odoo.
Let's look at a brief cost analysis.
Halo - $15,000/yr
Quotewerks or Zomentum $500/mo
QuickBooks or Xero $1300/yr
ConnectBooster $300/mo or more
Project management  $300/mo or more
ATS $5000/yr
HR system $150/mo/employee
Infusionsoft or Hubspot $1200/yr at least
Social media marketing  $200/mo
CRM - $300/mo
 
OR you could just stop all that nonsense.
 
Odoo. $47/mo/user.
Remember that this includes your website hosting too. And it turns out to be much better than WordPress, Joomla, or other smaller CMS.
What I find really hilarious is when I ask other business owners how much they are spending on all the components they use that spackle over the deficiencies in their PSA, they rarely know. It's like it is a financial hole in their business that they don't want to look at.
 
As of 11/22/2023, our 1 year Zoho subscription that we tried has been set for non-renewal. The primary basis for that was four wasted months of payroll, wasted time working with Zoho support, and wasted time working with Zoho consultants to try to get integrations with other modules in Zoho to work. The modules in Zoho One are designed to work independently. In order to get data to flow between them, integrations between the modules is required. We consistently found that those integrations between the individual Zoho modules did not work properly. We had other problems with it as well, but it became quite clear that Zoho One was not really an ERP because it is not foundationally designed with the premise that all of the modules are fully integrated automatically.
I looked deeply into Manage Engine ServiceDeskPlus for MSP also. I spent about a year on that investigation. I encountered a plethora of challenges with that and it still is a PSA-like mindset where ServiceDeskPlus cannot be a comprehensive business tool.
I encounter MSPs that use an outsourced helpdesk that requires the use of a specific PSA. I don't and won't outsource helpdesk for quality control reasons.
Overall, Odoo does everything better than ZohoOne. Odoo integrations are all there from the very beginning automatically integrated because it was designed as an ERP from day one instead of individual modules.
You can see a demo of Odoo at https://demo.odoo.com. Be aware that there are more modules available than what is shown in the demo, but the demo will give you a good overview. Odoo training is online and free. The documentation is online and free. Support is included with paid subscriptions and we have found that support is effective. Conversely, we rarely had any success with Zoho support. Odoo is more intuitive with things just working and being able to be figured out oneself through the use of documentation, training videos, and just playing with the software.
We use the Maintenance module which is good for a facilities maintenance team. I wanted my team to be able to log time entries against particular maintenance tasks. In 2023, it is not possible for time entries to be applied directly on maintenance tasks that flow through to timesheets for payroll. When I put in a feature request, with Odoo, they responded very quickly stating that they were aware of the limitation and were also aware of the need and value. With Zoho, I would put in a feature request and get a response in 9 months. 
I think that Zoho is so busy writing new modules that they have little to no developer time allocated to making the ZohoOne integrated ERP vision a reality.
We spent a LOT of time trying to use the recruiting module in ZohoOne and found it to be an exercise in frustration. We had a lot of success with the Odoo recruiting module with only a few limitations.
 
The bottom line is this. Find just one thing you can use Odoo for that can justify the monthly fee for one user. Get in there and start using it.
We have some clients who are using just one module for free. I got one client up and running on the project management module in a couple hours and got the client trained on it.
Another client, we put on the website module. The feedback we get from clients emphatically is that it is intuitive and easy to use.

Tech E&O and cyber insurance with Joe Brunsman

Tech E&O and Cyber insurance with:
Joe Brunsman of The Brunsgroup – Expert on Tech E&O and Cyber Insurance
YouTube channel – Joseph Brunsman
https://www.youtube.com/@JosephBrunsman
https://www.thebrunsgroup.com/
Damage Control book
https://www.thebrunsgroup.com/book2
Tech E&O and cyber
MSP should have a tech E&O policy. They cover different things. What types of third-party claims will they cover? A guy on the Que recently said that he did not think that E&O was required because his customers have never asked for it. You must have a TECH E&O policy.
What is the biggest thing that you need to pay attention into the E&O policy?
Look at the definition of technology services in the policy. Everything past that point, it does not matter if the definition of technology services is correct.
Avoid the named peril policy. An all risks policy is better. These are becoming harder to come by.
Named peril:  Technology services means:   there is a list
You have to prove to the insurance company that what you did falls within that definition.
What do you need to look for? “Including but not limited to”  contra proferentem = ambiguity is held against the draftsman. The onus is on the insurance company to prove that what you did was not covered under the definition.
How much coverage in the policy should they have?
How much cyber insurance do you need? Here are the variables that I think about. – See Youtube video
Brokers – There is no legal requirement that they understand or read the insurance policies.
Average IQ of an insurance broker is 104. They do not understand what they are selling. The onus is on the business owner to ask and to get the right things.
What is your major loss event? What are we worried about? Is that even possible to insure for those issues?
Step 1: Stop relying on the insurance broker.
Step 2: Fellow decision-makers in the business, what are you worried about? Talk to the broker about that. Then the broker finds “these are the options in the cyberinsurance market that address those concerns”.
Joe: Huge proponent of defense in depth over cyber insurance. Rank order the biggest bang for the buck. Felicia has been talking about that for years and is doing a webinar on 2/9/2023 on that very topic.
Insights from plaintiff’s attorney
Joe had a great convo with a plaintiff’s attorney and got his opinion on risk management.
Risk discovery question: What is the one thing that sinks the ship in the lawsuit?
There is an internal email. You knew you were supposed to do this. But they said it was too expensive. They were not going to do that. They understood the risk and just accepted it.
What could the business do in order to circumvent that email being a death blow in the lawsuit?
Plan of implementation.
No business has unlimited resources. No business is perfectly secure. You sit down the with business owners and MSP. We need to work on a plan to better your security. You don’t have unlimited money. I am a business owner too. You need a roadmap. Everyone signs off on it. We were trying, we were getting there.
Felicia: Wow this is astonishing because this is what we have been doing with clients for 20 years. It is the type of thing that a CISO knows how to do, but few others know how to do well.
 
 
Life hack tip from Joe:
Convo with the average business owner:
Obviously you are really good at what you do. You have built this business. Build a relationship with them. The MSP is not the subject matter expert on the client’s industry. Fluff their feathers. Transition that. I asked you a bunch of questions, thank you for hearing me. Now we are going to go through this. Can we just do the same thing in reverse? If you do not understand this yet, let me know and let’s break it down.
 
Joe and Felicia agree:
One way or another, those controls will be implemented. Read any breach notification letter. Magically we found more money to invest in cybersecurity.
Either work on your information security program monthly at a pace that your budget can absorb, or that decision of timing and magnitude will be taken away from you.

Implications of poor design on security - an example

Google and how they do their technology
Things that make security hard.
This is not an exhaustive list of the implications of poor design on security. Covering that topic adequately would likely rival the size of War and Peace. This is a discussion of a tangible example to convey understanding of how technology selection directly correlates to an organizations’ ability to secure or secure their overall environment. In order to accommodate something poorly designed, larger than necessary holes through security may need to be carved. Please get your CISO and security architect to perform a risk assessment technology BEFORE procurement.
Recent security news alerts discussed again why advertisements must be blocked. Google’s own ad network has been used for hosting and serving malware to victims.
Google and their netblocks
Their guidance to you is to whitelist their entire network blocks which is beyond insane. Just like the insanity of whitelisting *.windows.net which is what is advocated by some SaaS providers who host their resources on Azure.
Azure hosted customer resources are on windows.net. That means that a hacker can dial up a hosted VM and that’s on a windows.net FQDN and IP space.
You cannot just whitelist all of Azure either.
https://ipinfo.io/AS15169
Beware that software companies will put out idiotic statements in their support documentation that tell IT professionals to “open ports [range of ports] to all IP addresses contained in the IP blocks listed in Googles ASN.
Let’s be clear. Those are IP addresses not just for Google’s company internal resources. That is customer hosted resources that they don’t control, manage, or secure the content. So the Google netblocks represent 73.5 million domains.
There is NO legal defensibility in creating a hole that massive through any security system. Yet this is likely what 99% of IT professionals are doing because they are not network security architects. Business decision-makers must understand that there is a lot of bad advice that comes out of even major companies as it relates to information security risk management. They put out insane statements such as whitelisting the IP space representing 73.5 million domains.
Even if you look up a separate Google ASN, it is still 18,933,082 domains. That is clearly a massive amount more than just the small amount of resources that you legitimately need to access for something like Google reCAPTCHA to work. But because of the way that Google has designed their infrastructure, your ability to have network security is hampered.
 
https://chronicler.tech/firewall-considerations-for-google-recaptcha/
 
Autoblocking and DNS latency.
One of the major problems with using anything on Google’s infrastructure is that their entire system was never designed for compatibility with selective controls.
It was not mail.google.com. It was google.com/mail.
It was not drive.google.com, it was really google.com/drive. The real infrastructure was hosted as a subdomain of Google.
And then so many web developers have made google analytics a mandatory component of how their website infrastructure works that you have to allow it. It just allows google to be a data vampire.
 
Microsoft in contrast
https://learn.microsoft.com/en-us/microsoft-365/enterprise/urls-and-ip-address-ranges?view=o365-worldwide
 
There is a strong tendency to among IT support personnel to engage in over-troubleshooting. They follow software vendor’s recommendations and end up driving holes the size of North America through your security configuration. Please ensure that the personnel who are managing network security for your organization are actually qualified to do it.

Dark web monitoring and avoiding FUD decisions

Kathy Durfee – CEO & Founder of Tech House joined Felicia to discuss dark web breach monitoring
Scenario: FUD report from a competitor
Perceived: Multiple users in their environment were breached. Perceived proof was report with the listing of the users and the passwords and columns that the customers did not know what that data was.
Good: Customer told their current IT service provider about the report.
FUD – Fear, Uncertainty, and Doubt – is, in the wrong hands, a powerful tool to drive snap decisions within a company. However, it is not a viable or valid sales tactic: for all it could potentially do well, causing unnecessary stress and suffering is what it does best. Speaking with Kathy Durfee, CEO and Founder of TechHouse, a managed services and solutions provider based in Florida, we walk through a recent case of FUD with a customer of hers that received a worrisome report from a potential competitor. During our chat, we covered:
The key aspects of FUD (and how it does not work)
What the Dark Web is, and the logistics of monitoring and combating it
Leadership training and best practices for helping a team best meet their security and regulation requirements
Identifying the key differences between commodified and relational partnerships, especially in the technological sphere
Shared responsibility between MSPs, their customers, and those customers’ clients
Where does dark web monitoring and dark web data risk reside on the continuum of risk? How best to mitigate?
What really is the risk and the mitigation?
Put the efforts into prevention.
Put the individual in the driver’s seat of managing the risk that is best managed by them by putting the right tools in their hands.
Resources
https://haveibeenpwned.com/
Perception of the proper allocation of the budget
Businesses must make time for training.
ITSP must include in service catalog what the client is getting in terms of services.
What do we need to do? Cross reference on tools that accomplish outcomes and cover risk mitigation and ensure that the client understands what those are.
Training is how you squeeze the juice out of the orange. Without it you may not get all the juice out of the orange or get any juice out of it at all.
Common business objections to allocating time for training
Payroll costs, but avoiding training is not legally defensible anymore.
Policies
The IT Service provider CANNOT alone write policies for you, and they CANNOT approve and enforce your organizational policies.
Four pillars
Policies
Technical controls implemented
Automation of technical controls
Reported to the business – It’s YOUR report, your organization.Shared responsibility – some months the CFO does it, some months the CEO does it.Set a schedule and do it. 3 weeks any habit; trainer or partner
Do you look at your P&L and balance sheet every month? You should be understanding the reports from IT.
 
An interesting lawyer opinion on the topic:
https://abovethelaw.com/2023/01/dark-web-monitoring-for-law-firms-is-it-worthwhile/

The relationship between proper data handling and real risk reduction

Those who listened to the November 19th, 2022 podcast I did with breach attorney Spencer Pollock know that he stated that 90% of the breaches he was involved in over the prior 12-month period would have been non-reportable had the data been properly encrypted.
https://qpcsecurity.podbean.com/e/what-you-must-do-in-order-to-prepare-for-a-breach/
(Review link above for attestation and regulatory enforcement proof.)
I have three major points for you in this show.
You need an IRP
You need a CvCISO
And you need to understand how data is being handled in your organization
Let’s first talk about CvCISO
Help you understand why you need a CvCISO working with you on a regular basis because even if you are a really large organization, the probability that all of your processes are clean, secured, compliant, and all your end user training is effective, well that probability is not high.
https://qpcsecurity.podbean.com/e/understanding-vciso-services-and-why-you-need-them/
Incident response plan
Virtually every organization is now required to have a written incident response plan. These are some examples of people that must be specifically listed in the IRP. What does your organization do when they don’t have these people as full-time internal staff? You need a CvCISO.
People you are required to name explicitly as part of your incident response plan:
IT technical staff
Incident response manager (this better be a CvCISO or a certified incident response company)
IT director
CIO
Stakeholders such as board of directors and heads of business units
Finance director
Communications manager – this is either your internal PR person, your internal corporate counsel, or your breach attorney
Legal representative – either your internal corporate counsel or breach attorney
Human resource manager
Types of data
Let’s talk about some real-world examples of data insecurity. Let’s start by establishing what some categories of data are. PII, PHI, PCI data.
PHI is personal health information so think of that as drug screening results as well as medical records. So it’s not just healthcare organizations that have it. Anyone who does drug screening will have PHI.
PII is personally identifiable information such as your name, contact information, social security number, I-9 information, a copy of your passport or driver’s license, and non-public photos of you. This is also your direct deposit bank information. I would also include your salary at your job is PII. It is certainly non‑public. So who has that kind of information on you? Well anyone who does HR recruiting or has employees is typically going to have this kind of data.
I encourage small businesses to use a PEO and not store any of this data themselves. They should outsource that entirely. Some HR management firms have areas in their SaaS platform that their customers (your employer) can upload documents to and store them securely and NOT on the employer’s environment anywhere.
PCI is payment card information. If an organization processes credit cards in any way outside of a contained e-commerce and merchant processing platform, then they probably have PCI data that is on a system they control. Many retailers just use SaaS apps that directly integrate with merchant processing to avoid any storage or holding of PCI data. You should expect that larger organizations are retaining your credit card information.
Applicant tracking and employee onboarding systems
The security of these systems is only as good as the security of the company that is using them, their processes, how they handle the data throughout the flow, and how documents you complete for them are disposed of if they were submitted in paper format.
Good process
As you interact with the recruiter or prospective employer, all of the data goes directly into an applicant tracking system that is SaaS by the applicant themselves. The only thing that may be emailed would be a resume. Any assessment results or applicant data is all direct input into the ATS. The ATS is SaaS cloud hosted with a very secure company and all accounts which access the data are on a need to know, RBAC approach with MFA enforcement.
WOTC information is all submitted by you directly into the WOTC company website
All of your PII would be submitted by you directly into the HR enrollment/payroll system without intervention from anyone else. No one else needs to handle your data.
The data you submit is only being submitted to a high security SaaS HR management/payroll platform.
Your employer never needs to download and retain any of that information because it is stored in the HR management system. Nor did your employer ever need to have a copy of the information you submitted because you submitted it yourself. So you know it is not in their email or on their servers anywhere. They also did not print it and then not shred it.
Bad process
You are an applicant and the company you are applying to has you fill out paper forms. You do and then they scan those forms with a scanner and send those files somewhere. Let’s say they are scanned to an insecure location on the internal network. Then someone retrieves the scanned images of the paper you filled out and emails them to a distribution list.
So let’s go over what is in the scanned PDF file that got emailed to an internal company distribution list.
Direct deposit information – full banking account and routing number
Full name and address
I-9 verification which includes social security number, driver’s license number, and birth certificate
W-4 which contains PII and SSN again
Copies of your signature
Date of birth
Your offer letter including salary and benefits
Concerns
What happened to the physical paper copies of the forms you completed? Were these shredded same day?
Was the information in the email distribution list forwarded to anyone who did not have a complete need to know?
Was the information forwarded to a party external to the company?
Document management platforms
Premise databases often have a lack of encryption
Lack of data encrypted at rest and quite possibly the data is not encrypted in transit. If the system that the data is stored inside of is a premise-based thick client application such as an application that has SQL server as the back end, it is not likely that those communications between the thick client and the database server are encrypted in transit. The SQL server most assuredly is not encrypted because very few applications support SQL database encryption and even fewer IT people know how to set it up.
I have seen document management platforms with 500,000 records in them containing some of the most sensitive PII and this data was not only housed on servers that were unpatchable and fully deprecated, but the data being transmitted to/from the server was not encrypted, nor was the data in the database encrypted at rest.
If you put a dollar figure to the cost of a breach and it is associated with the number of unique records that contain reportable information, the cost of that old, insecure server just went through the roof.
Even if you say $1/record, that is $500k. Wowzers! And it’s not likely that was the only server compromised in the breach.
What data is stored in people’s emails when a company does not have solid policies, end user training, and technical enforcements to prevent the data from being improperly stored?

Understanding vCISO services and why you need them

Recent question I got:
What are the major changes that you have seen from security auditors in recent years and/or where do you see the audit process heading?
Quick response:
For the sake of a high level, automation is and will continue to be used. The size of the IT service provider is NOT a conveyance of their capabilities or capacity.
Many 60 person MSPs are grossly incompetent. Some small teams of about 8 people are exceptionally skilled.
C-suite needs to drive it from the end in mind. The end is compliance attestation. Back into it from there and ONLY use a team which also has the technical capabilities to perform the remediations.
Do not use vCISO services from one company and remediation services from another. You get too many cooks in the kitchen and a disjointed and more expensive outcome will be the likely result.
 
The insurance companies are pushing the cost of the audit on the insured or applicant. This will involve eating tools and processes that connect with their assessment process.
Hence why it is crucial to work with a company like mine that has these workflows. Most don’t.
 
In this podcast, I provide an overview of the role of executives, managers, internal IT, and the CISO in business risk management. Until all parties understand that this is not information security risk or cybersecurity risk, it is business risk that they are responsible for managing, then it is not likely the situation will improve.
In order for business risk managers to make good risk decisions, they first have to engage and be involved. They cannot put their head in the sand and believe that "It's an IT problem." No it's not an IT problem. When the HVAC system is open for hacking to everyone on the planet because the facilities director refuses to collaborate with IT security to come up with a solution to maintain business functionality while managing risk, that is a business risk issue.
If the facilities director REALLY believes that it is an IT problem, then IT needs to be provided the authority to rectify the issues. And when the facilities director's access is interrupted, then they will be forced to engage and collaborate at that time. But executive management needs to have the intestinal fortitude to enforce policy. The policy that IT does have that authority and no IT will not be retaliated against. That is one approach. The other approach is that the facilities director needs to acknowledge that THEY are responsible for business risk management of the HVAC system. So if the facilities director wants the right to complain when their access is revoked, then they cannot abdicate their responsibility and accountability for the security of the HVAC system.

What you must do in order to prepare for a breach

Breach attorney, Spencer Pollock joins Felicia for a vigorous discussion of what you must do in order to be prepared for an incident or breach. Learn from the breach attorney perspective. 
Spencer is with the well-known firm McDonald Hopkins. 
Policies
preparation
incident response plan
tabletop exercises
must get breach attorney involved before there is an incident
determine your team in advance
What's new?
regulatory enforcement
multi-state class action lawsuits
attorney generals getting together to class action effort
Regulators DIG
They want to see your policies.
You must demonstrate your administrative, physical, and technical controls.
Attestation proof of state is mandatory
You better be able to enable your breach attorney to tell a legally defensible story.
How many data breaches could have been avoided by properly encrypting the data? - 90%
 

Information Security, Cybersecurity, and Everyone’s Responsibility

What is information security versus cybersecurity?
What are policies and why do we care?
Isn't that IT's problem?
Examples to learn from

Ripping apart cybersecurity insurance

Special guest:
Vince Gremillion – President and Founder of Restech: CISSP, CvCISO, GCIH
Overview
Travelers policy – requires MFA on switches. They require you comply with the intent of that.
Recent Cowbell application did not require MFA!
What is required is contingent upon the coverage you are asking for.
Some suggestions:
Never fill out an app for a client, not even partially
MSP comms to a client should be in a document in a detailed format and it should be digitally signed and locked for editing through that digital signature. I use Adobe EchoSign for that.
I address everything in a CRAQ format and then include for the client a spreadsheet which is a cross reference. I will never answer any of those questions on the application directly because I can tear holes in every single one of those questions.
I reject many of those cybersecurity insurance application questions as yes or no. Yes/No just does not fit.
All the insurance carriers and underwriters have accepted my method which I fine to be the only defensible approach since yes/no is inadequate and does not protect the insured/applicant or their MSP.
Future strategy
This is exactly why we need CISO platforms which have automatic data ingestion and transmission of the data to insurance carriers in standardize pre-scored format.
Check out this podcast on the topic: https://qpcsecurity.podbean.com/e/ciso-workflows/
Business owners: You own the risk, you decide what to do with that. If you did not vet the MSP or the vendor or their stack, that is ultimately your risk problem.
HUB International as a broker specifically tried to suggest to one of our clients that the MSP should be filling out the cybersecurity insurance application. I found working with HUB International to be very difficult. Marsh McLennan Agency https://www.marshmma.com/ was very good to work with, but they cater only to larger employers.
Gem from Vince: Compliance as a threatIf law firm A can no longer do business with customer B because they don’t have compliance, that is a threat.

CISO Workflows

Frank Raimondi, VP of Channel Development at IGI Cyber Labs
IGI CyberLabs has a product called Nodeware which does continuous vulnerability assessment.
PenLogic – regular penetration test – once a quarter deep dive heavy one and a monthly light test.
CEO buyer’s journey
Security velocity
Risk scoring is part of security velocity
Improve your cyber-hygiene – all small businesses
Security 101 is inventory 101
Cysurance – warranty and liability company
It’s good that insurance companies are trying to be more objective about the real risk metrics. Get the scoring and get the data about how risky they are. This feeds into the evaluation data which is used for underwriting.
FTC Safeguards policy impact
Operational security issues – MSPs that post all their personnel information publicly.
The impact of customer contracts and compliance. Squeeze between cost and staying in business in terms of insurance and customer contract requirements.

Business Email Compromise

Ken Dwight is “The Virus Doctor” – Business consultant and advisor to IT service providers and internal IT at many businesses who have come to him for his training, has his own direct clients. Ken conducts a monthly community meetings for alumni. He provides a list of curated items of current interest for discussion and resources, and has a featured topic which often includes another speaker to provide breadth of perspective. He has been doing this community service for 83 months!
I asked Ken to cover with me some topics that from his perspective don’t get talked about enough.
Business Email Compromise
Also known as CEO fraud. Impersonating a CEO for purposes of wire fraud. We are focused on the technological solutions. There is no technological solution for eliminating BEC.
CEOs must be part of the solution.
Example: Subcontractor to Airbus. Used to dealing with multi-million-dollar wire transfers.
BEC is a large Fortune 500 issue, it scales down to one user environments.
Title companies are a big target.
Retention policies and standards for WHERE to store what kinds of data to make sure that email is not a file server thereby increasing the risk of what data is compromised as part of BEC.
Perfect example of the beginning of an incident response plan or a tabletop exercise. Orgs must define the cost of compromise. That plan needs to be in place long before. It makes a recovery so much more straightforward.
Attackers analyze their victims in tiers. Potential victims $10 - $50mm revenue organizations. Reputational damage, but not big enough to have an adequate cybersecurity budget.
ShadowIT is a problem, which is why you must address it with a CFO-enforced procurement policy.
Proactive management of M365 tenant security configuration is so critical
The security of your tenant is not included in the fee for biz premium or the overall licensing.
How much activity there is, changes, products, services, vendors. Ideal stack, layers, point solutions within that. Revisit that in a period of time like a year.
This is a nice resource for M365 security and BEC.
https://www.blumira.com/office-365-security-issues
Direct advice from Ken
One topic I believe falls directly into this category is the issue of Business Email Compromise, as opposed to actual malware / hacking / ransomware attacks. As you know, the losses to BEC still represent a greater dollar value than ransomware, according to the FBI statistics. But BEC isn’t even a technology problem, it’s pure social engineering – and no additional layers of hardware or software “solutions” will prevent it or reduce the cost to its victims. In my opinion, that’s why you hear so little on the subject from the cybersecurity vendors.
Another topic I find interesting, but haven’t really heard any vendors or industry pundits talk about, is the whole new ecosystem and infrastructure produced by modern threat actors. The whole business model of these sophisticated criminals has created occupations, titles, and job descriptions that didn’t exist a few years ago. Some of these are a result of the specialization, compartmentalization, and outsourcing by these organizations; here are a few that come to mind:
Breach attorney
Ransomware Negotiator
Initial Access Broker
Cloud Access Security Broker
Multiple “As-a-Service” offerings:
Ransomware as a Service
Phishing as a Service
C2 as a Service
Another area that is mentioned fairly frequently, but typically fueled by more heat than light – and raised as a point of frustration by MSPs and IT Solution Providers in general – is the users who still believe they don’t have to worry about cybersecurity, hackers, malware, or ransomware, because they “don’t have anything the criminals would want,” or words to that effect. I believe those users need to comprehend how real and serious the threats are to their business.
By defining the multiple tiers of threat actors, the threat vectors they may employ, their potential victims, the assets owned and managed by those victims, and the attacker’s strategy for monetizing those assets, I believe it becomes obvious that every organization and every individual is the intended target of some subset of those threat actors.
Visit this resource for help making argumentation. Ken is working on some additional materials for end user cybersecurity awareness training.
https://qpcsecurity.podbean.com/e/the-real-reason-you-cannot-afford-to-have-a-cybersecurity-incident/
 

Vulnerability management with Felicia and Dan - Part 2

This episode of Breakfast Bytes is Part 2 of a series where Felicia King and Dan Moyer of QPC Security continue their conversation on Vulnerability Management. Listen to Part 1 at https://qpcsecurity.podbean.com/e/vulnerability-management-part-1/. 
In today’s episode, Felicia and Dan discuss vulnerability management workflows, supply chain risk management, starting with security on the front end rather than retrofitting, and proper patch management. 
Workflow management 
01:10 CISO-related (Chief Information Security Officer) workflows are at the core of what is today’s necessity, and we will only see it become more mandatory within the next couple of years. Organizations that do not have vulnerability management workflows in place in a comprehensive way are going to find they have too much technical debt, deferred maintenance, or deferred security to be able to dig themselves out. This won’t be from a lack of money either, but a lack of manpower and time in the day to rectify the issue. 
Supply chain risk management 
02:43 SaaS vendors have vulnerabilities and very few of them have in their contracts your rights and their obligations. What kind of questions should you be asking your SaaS vendors that in many cases you are responsible for as an organization? Here are just a few: 
Do they have continuous vulnerability management scanning going on with regards to their SaaS platform? 
How are they classifying vulnerabilities?  
How quickly are they going to resolve vulnerabilities?  
How are they communicating these issues to you?  
Do they use API security scanning?  
How do they adhere to OWASP API standards and best practices?  
What are they doing for you in terms of supply chain risk management or software bill of materials? 
Your organization’s CISO or vCISO should be in your court getting answers to these questions if they are not being addressed by your SaaS vendor or addressed in your contract. Having a proactive, highly functional, highly communicative, and open, honest working relationship with your CISO will ensure you have the protections your organization needs. 
Proper patch management 
04:51 Let's walk through an example of patch management in an environment with Hyper-V hosts, Dell PowerEdge server, domain controllers, business critical SQL servers with essential business applications, virtual machines, remote sites, on-site and offsite backups, hardware at different speeds, and then all these third-party software on these workloads – how do you patch all these things? 
06:11 It is exceptionally important to note that some patches will step on or over each other, be required to be put in place and rebooted first, and then other patches applied on top of it. The time it takes to patch a server can be exacerbated by trying to accomplish, say, five patches in one changewindow rather than one patch/reboot followed by another one patch/reboot, and so on. 
07:48 Watching the servers reboot is an important piece to verify the workload comes back up reiterating the point made in Part 1 of this series that adequate patch management of an entire server for $50/month cannot be done. 
Domain controllers 
09:19 There tends to be multiple domain controllers or, in the case of just one, it has been designed so that it can reboot whenever it needs to allow for patching. The domain controller is the brain of everything, and since it can reboot whenever needed to apply patches, it can facilitate that while staying available when everything else comes back up. 
Typically we will start with domain controllers as the first thing patched and verified. Now if there are multiple, and depending on how critical the environment is, a rolling out patch might be done so that these secondary domain controllers or ones that are not on the best hardware are patched and then they sit for a period. 
Backup plans and backstops 
11:29 Part of that patching methodology is your backup plans and backstops – having the tools and everything else in place to uninstall a patch if needed. When we set up our servers, we always have Command Prompt and PowerShell already queued up on those devices when we log in. Then we have the availability of pre-planned scripts that we can adjust as we go but most importantly, all the tools are there and available.  
Importance of roles on servers 
12:25 Part of your ability to have resiliency in the environment is the ability to reboot whenever you need, because you have redundancy and resiliency. Because it is a single role server, it gives you that agility to be able to resolve and prevent issues.  
Therefore, workload design is the name of the game. Whatever you think that cost is of that additional virtual machine, that is nothing compared to the problems that you cannot solve because you tried to shove a bunch of stuff together in workloads that did not meet because they were mismatched workloads. 
Many patch managers are not comprehensive and there is a lack of consistency in of what is getting patched on a well-designed domain controller versus a third-party party application server.  
Physical servers 
16:09 Watching a virtual machine reboot while maintaining efficiency and not biting off more than one can chew is crucial, but we are also finding is increasingly important to watch the physical servers and that can only be possible with the right hardware. 
How are you auditing and confirming that patches are being applied and which ones have not? At QPC Security, we bring all the virtual machines down and reboot the host as a prerequisite for patching because it gives you a clean slate to start your patches. Then we will use the patching methodology to push specific patches down to it. We use our patching piece to push specific ones because not everything is needed for hosts and other pieces that we have identified will cause an issue, is a multi-patch, or a multi-patch/multi-reboot process. 
Taking one step at a time, pull it down, apply patches, make sure everything is happy coming up. Go through that entire process again. While we are connected to iDRAC, we watch the server, reboot, apply patches, come back up, make sure all the VM's are checking in properly, we are making sure everything is available, then they go through that process two to three times. It depends on how many patches are available and what things got pushed out. 
Everything has patches 
20:39 If you have a hypervisor that is not giving you patches; you should not be using them. Likewise, if there is no product improvement then there is no security management from that vendor. There is no easy button or a set it and forget it. 
21:42 When IT is not confident in how a process is going to work, they do not want to touch it and that is exactly where a vulnerability arises. Say a consultant installs Cisco, but without a brand expert or budget in place keep the consultant to maintain it, it remains unpatched and therefore vulnerable. That is precisely why organizations need to have a business continuity and disaster recovery (BCDR) plan in place and a procurement policy that drives effective vulnerability management. 
Incremental patching 
25:26 When people are too afraid to patch the hardware, it does not get patch which accumulates over time in terms of technical debt and the technical issues it accumulates. Attempting to patch too many patches at once or jump too many versions results in the reboot cycle of death or a very time-consuming reboot because you are not running a vetted, tested, and supported configuration. The more time and versions you allow to pass between patches, the more divergent from manufacturer’s tested config those updates become. 
Buying the right hardware to begin with saves you money down the line 
33:20 A crucial piece to vulnerability management in your workstations is BIOS, drivers, firmware. If you buy the right hardware to begin with that has the automation engine built into it and when you deploy it you are configuring it accordingly, it becomes far less expensive than paying a human being to manually babysit your vulnerability management. 
Not all workloads are created equal 
34:59 A word of caution when an IT service provider quotes patch management for your organization. When it comes to patching business line apps that need high uptimes because it costs a business thousands of dollars per hour to be down, what patches does the ITSP apply and with what preparation for back out plan?  
In many cases, an ITSP is giving a client the perception of patch management, certainly not vulnerability management, but in reality they are simply doing a Windows update and only some third-party patching, which might only be five third party applications. At QPC Security, our catalog of patches of over 9500 software titles that we are patching and there is no automation. Visit https://www.ivanti.com/partners/ivanti-software-catalog to learn more about the normalization of software titles. 
Cybersecurity insurance applications require continuous vulnerability assessment and vulnerability management. However, most IT service providers do not offer comprehensive patch management. Their vulnerability management claims are grossly misrepresented to the point of malfeasance. 
Vendor documentation & software bill of materials 
37:43 You cannot keep your head in the sand – all these things must be considered when receiving a quote from an IT service provider.  
In cases when the software vendor is not offering competent documentation, your organization must rely on the legwork of your IT service provider to offer timely patches at opportune times. Do not forget that many ITSPs will charge you to run patches on the weekend or evenings when there will be minimal impact to your business.  
"Titrics"
43:02 Your ITSP should have vetted and tested procedures and protocols for implementing patches, yet all too many do not. So many times, we see the priority of IT companies are how quickly they can close a ticket and rely on the software companies to do it for them. This focus on first-call closures and ticket metrics (termed here as “titrics”) is grossly underserving their clients and their clients’ organizations. Proper documentation allows for better time management and to offer effective support to best serve the needs of the clients without requiring the assistance of the third-party software vendor. 
47:05 Gaps in change management, change control, and documentation for server workloads arise when an ITSP is focused on ticket-based productivity rather than quality of service. The original scope of the project by the ITSP requires evaluation from someone who can accurately evaluate the needs of the client’s organization. When the bid is too low, the needs of the client are not going to be met, the work will not be completed, and the organization is left vulnerable. 
50:03 Unfortunately, an incompetent ITSP will leave out what services they had to cut out on the race to the bottom of the pricing model and that leaves it up to you, as the business owner, to be aware of your organization’s cybersecurity insurance policy requirements and how they are being fulfilled. 
Questions? Reach out to us
QPC Security proudly serves businesses with virtual CISO services for our clients. If you are interested in learning more about how QPC Security can serve the needs of your organization please visit https://www.qpcsecurity.com/ or call one of our experts directly on (262) 553-6510.  
Stay up to date on the most recent episode of Breakfast Bytes by following the podcast on Podbean at https://qpcsecurity.podbean.com/. 
 
Learn more: https://www.complianceforge.com/faq/word-crimes/policy-vs-standard-vs-control-vs-procedure
 

File integrity checks (hashing) versus communications or data encryption

We have seen some really goofy cybersecurity insurance application questions. It is always best to not answer a question that is goofy, but instead to write an addendum that defines terms and explains the cybersecurity posture of an organization related to the topic. You need to try to figure what the insurance company was trying to evaluate rather than just answering their questions because their questions are frequently not suitable for yes/no answers.
Greg Cloon joins me to discuss this topic.
We also touch on when you would use file hash integrity checking, when to use disk encryption, and when to use encryption for communications.
Here's a link to IISCrypto.
https://www.nartac.com/Products/IISCrypto/
 

Vulnerability management that every business decision maker needs to know about - Part 1

Felicia King and Dan Moyer of QPC Security talk about vulnerability management, patch management and all the things that business owners are generally not understanding adequately. As a result of that, you're being underserved, misled, and in some cases were lied to and ripped off.
Ultimately, many business owners are refusing to pay for what they need for adequate risk management because they don't understand what they need. In today's episode Felicia and Dan fill that gap. 
Announced on October 6, 2021, the US Department of Justice Civil Cyber-Fraud Initiative is applying the false claims act to those who:
fail to follow required cybersecurity standards
knowingly provide deficient cybersecurity products or services
misrepresent their cybersecurity practices or protocols
violate obligations to monitor and report cybersecurity incidents and breaches
Just let that sink in for a second. So, is your IT service provider really meeting that standard? I sincerely doubt it.
01:23 The difference between vulnerability management and patch management 
Holistic vulnerability management includes, but is certainly not limited to: 
Software bill of materials analysis 
Supply chain risk management 
Third-party risk management 
End-of-life software 
Asset inventory up to date 
Lifecycle management 
Continuous vulnerability assessment 
Frequency penetration tests 
Tabletop exercises 
Procurement policy 
04:38 Cybersecurity insurance applications aren’t asking JUST about patch management 
When did you have your last penetration test? 
Do you have continuous vulnerability assessment in place? 
How long are you going to go without having the patches applied in the environment? 
If you think adequate patch management can be done for $50/mo/server, you are hallucinating.
So, what’s included in patch and vulnerability management?  
05:34 Patch management 
Patches are the building blocks that are improving the software that lives on the hardware. Without software, you can't interact with the piece of hardware unless it's purely mechanical, and even then there's still improvements of usage.  
How do you manage and protect those tools of your business from threat factors? 
09:20 Third-party patches & vulnerabilities 
IT service provider proposals are telling business owners that they can patch their servers and their endpoints and automate Windows updates and some third-party patches. What are those third party applications? What about all your custom business line applications? Do you actually want your critical SQL server to have its SQL instance updated using automation? How much money does it cost you if that workload is down?
10:27 Asset management 
Do you know what you have in your environment? Do you have accurate asset management and vulnerability assessments? Simply stated:  
“You can’t secure what you don’t have an accurate inventory for.” 
It is a regulatory requirement and cybersecurity insurance requirement to adequately document and understand software dependencies in your environment. That requires a proper inventory of your hardware, software, and subcomponents of the software. This is frequently referred to as SBOM - software bill of materials. And if you think your software vendor is going to provide that information, please go ask them for that information. You will probably get a blank stare. IS security engineers can figure it out on their own.
18:48 Implementing proper procurement policies 
Does your procurement policy support your vulnerability management strategy? Does your software acquisition and implementation policy (if you even have one) support your cybersecurity insurance and regulatory requirements?
When business decision makers put pressure on an IT service provider or internal IT to implement new software without proper security protocols, vetting, and process documentation, vulnerabilities are nearly always introduced into your environment. Sometimes that comes directly from their insecure software. Sometimes it comes from the tools and connectivity they use to remote into your systems or things like API connectors that your IT is supposed to just blindly trust the software vendor to secure their software with zero validation or proof. A proper CISO on your team or through your ITSP will be able to directly vet the vendor and software itself.
You are required by cybersecurity insurance and Federal regulatory guidance to do so. It is also in your business's best interest to do so.
Be very careful looking for just certifications for someone who says they are a CISO. The majority of CISOs do not have technical chops. They are often compliance managers that cannot do the technical work. Those people have limited usefulness and will not be able to 
All of the vCISOs at QPC are hardcore technical because we understand the essential nature of that skillset being a mandatory requirement to deliver effective CISO services.
20:24 Privileged access management and privileged password management 
How do you know who has access to remote access to your systems? How many people will have access to your systems? Today, there are many IT service providers who are not disclosing their outsourced Helpdesks that are giving full administrative-level access to a customer’s back end to all those workers at the virtual live Helpdesk. Most ITSPs also fail to disclose the totality of the quantity of people that will end up with admin access to some or all of your systems.
Ask yourself. If you have 25 office personnel, why would it take 30 remote people to have admin access to your systems in order to provide competent support? Do you think it is actually possible to have a high security environment and magically keep 30 people fully up-to-speed on the exact correct configurations required in your environment and what the interaction effects are? It's not possible and will never happen. 
24:27 A procurement policy can keep a business' IT costs stable 
The number one thing that business owners complain about is the cost of maintenance. With a procurement policy in place and by working with their IT service provider and procuring anything that they do not have a full understanding of the total cost of ownership for – costs can be managed. 
Does your procurement policy support your business strategy and needs? 
34:22 Understanding the cost and time of device and software procurement 
There's also a lot of other risks that the vast majority people don't think about; they tend to only think about the budgetary risk. However, getting the strategic input from a CISO or CIO to develop an understanding of the minimum pricing floor and how that affects the total cost of ownership, can save a business not only money but time.  
SaaS can get you closer to a flat-rate cost but you may have inherited additional risk and vulnerabilities, depending on how the new technology interconnects with your systems. Additional risk factors are:
counterparty risk
structural increase in cost of doing business risk
accessibility risk (redundant access is then required and cannot be fully mitigated)
external software vendor attack vector risk that cannot be mitigated through Layer3 ACLs
takedown/contract risk
37:33 Cloud vs on-prem security 
It's still a fallacy that having your systems in the cloud is better and cheaper, incorrectly thinking they can have as good security in the cloud as they can on premise. Going to SaaS can provide a lower and more predictable TCO if the counterparty risk you accept is worth it. But picking up your servers and hosting them on someone else's infrastructure will never be less expensive. IaaS cost savings are a fallacy for the majority of businesses. The exception being massive companies with heavy DevOps needs for spinning up and down workloads quickly. Most of those items are being migrated to Kubernetes and OpenShift.
46:48 IT/IS is not a utility 
The electricity company, the water utility, garbage pickup, fire and safety, ISP – they are monopolies and uni-taskers. Whereas IT is far more complex. People tend to think that if it’s a utility, therefore it’s a commodity, and if it’s a commodity it doesn’t matter which service provider I choose. 
Business decision makers are trying to manage budget risk without understanding their requirements. They also want to have budgetary control while abdicating their involvement upon outsourcing their IT to an ITSP.  
An IT service provider can be a partner to success and can help businesses develop better business strategies IF there is regular and open communication.  
 
This is part 1 of a 2-part series on vulnerability management. Listen to Part 2 at https://qpcsecurity.podbean.com/e/vulnerability-management-with-felicia-and-dan-part-2. To learn more about QPC Security, visit us at https://www.qpcsecurity.com/
This is another resource for vulnerability management information.
https://land.fortmesa.com/vulnerability-management-101
 

Signs of insufficient networking knowledge

Scenario 1
Phone VLAN on a switch and cross connected into a Firebox with desk phones, PCs, and printers in the environment
Questions we actually got:
On Monday, we send over the list of what switch ports are for printers, which are for PCs, and which are for desk phones. Technician says that two of the three phones are not working. We use our awesome switches to find out exactly where these other phones were plugged in. The phones were plugged into the wrong switch ports. Move desk phones, phones work.
Then later, the technician runs a test for the VOIP service from a PC on the PC VLAN not from a PC connected to the phone VLAN. So the test for the VOIP service fails. Security zone profiles exist. It is not acceptable to have an allow everything network security posture. Configures needed to support desk phones are completely different from those that are required to support domain joined Windows computer assets.
 
Some ITSPs have to pay for expensive add-ons like Auvik to try to compensate for the fact that they have inadequate switching equipment with inadequate design and a sprawl that they have to inventory and keep track of. TCO comes from how much time it takes to maintain, manage, adds/moves/deletes/upgrades, troubleshoot. If I have to physically go to a site to chase some cabling, something is really wrong.
The technician in this scenario also could not believe we wanted two network cables between the switch and core router. They are not the only one. I encountered this lack of vision of understanding in another client IT director earlier in the year. If you don't know why you would have two network cables between a switch and a core router, go figure that out.
Scenario 1
Phone system with desk phones. Each desk phone has its own network cable, which is good. Phone subnet should be a separate VLAN, but the choice is made by ITSP to separate the phones using physically separate switching equipment. That is something I would never do.
Commentary provided by ITSP:
I don’t like VLANs. I would rather setup a network with physical segmentation. Results in:
Loss of visibility
Loss of network resiliency
More expensive because you have more switches to babysit and troubleshoot
So if you have 20 or 40 VLANs, so does that mean you are going to have 20 or 40 physical switches?
If you don’t have 20 VLANs then what network security do you really have?
How do you present virtual servers on the proper microsegmented security zone when you cannot transmit tagged packets?
Let’s just talk minimum VLANs that we typically see here:
SwitchOOBM
ServerOOBM
SwitchMgmt
WAPMgmt
Phone
Surveillance
CorpWired
CorpWireless
GuestWireless
HVAC
ElecMon
Chromebooks
CaptivePortal
Tier0
DCs
AppGroup1
AppGroup2
DeprecatedApps
Printer
Storage
IAM
RMM
Clearly anything over two becomes ridiculous to do with physically separate switch equipment. The days of this paradigm or strategy are long gone since cybersecurity compliance is requiring microsegmentation. And network security strategies and technical controls are some of the most effective primary and compensating controls for cybersecurity posture for all the protected assets regardless of type.