Is this the end of AI? (Analog Intelligence).

The past few months I’ve been quietly watching LinkedIn and other social forums, keeping up with the latest trends and news. One surprising finding is the amount of content written or contributed by AI (Artificial Intelligence).

It started with a new CISO asking for some advice on how to write policies for her new role. I waited with bated breath for the replies. But soon came the non-helpful content like “buy generic policies from xx website”, but soon came the words I feared would appear on LinkedIn, “just use ChatGPT” other AI tools are available. Then followed soon by a comment written by AI with some interesting suggestions including uploading all your policy, process documents and handbooks allowing ‘THE TOOL’ to design policies closer to your requirements.

Whilst its clear there is a place for CHATGPT whether this is the free or paid version but its worth understanding the risks before you start throwing your documents or client related documents into your choice tooling. For the benefit of simplicity, I will use CHATGPT for understanding the risks. (for those keen eyes people yes, this information has been populated by CHATGPT.)

Below outlines some of the poignant terms and risks related to firstly using the free version and secondly the risks of uploading confidential or sensitive documents:

This led me on a deep dive into LinkedIn and Reddit, I recall reading that the best way to get quick answers to your specific query is to publish the question and setup a second account with a different name and answer wrongly as people are always quick to correct incorrect answers but here is where I found this to now become incorrect.

Some interesting statistics based on my research – Over 90% of replies asking a technical question or a senior management technology issue had some content which can be analysed as “AI-style writing” looking at tone, grammar and even sentence structure detecting to a high degree of accurately. That’s not to say that all the content was written by Artificial Intelligence but does demonstrate the amount of content being published with some content not Human Written.

Luckily diligent companies are limited or restricting access to these tools reducing the risks of misuse, and the knock-on impact of the regurgitation of incorrect or misleading information. This is not to say that everything the tools spit out is incorrect.

So back to the rabbit hole I go…

I started to look through many of the free document resources online sharing policies and other documents you can “use” for your organisation. This led me to putting these documents through AI detectors 2 things stood out, some of the documents had commercially sensitive, client or customer data, company specific details in them. (don’t worry I made contact with all of them I found and some have been removed or edited).

Over 80% of the documents I reviewed had Artificial Intelligence written content. Even more striking was the amount of content reused in many of these documents including mentioned to old standards and regulations including but not limited to The Data Protection Act 1998 (DPA 1998) which was replaced in May 2018, ISO27001:2005 replaced in 2013 with ISO/IEC 27001:2013 and some other examples of US standards.

One size fits non

For those writing policies for the first time its worth understanding why you need the policy and who its being written for, the use of tools can give you a feel for the type of content and styling being used but it does not know your audience, your company, your challenges you’re facing or even the standards the policy is expected to meet.

Tooling just isn’t the right tool. If you already have other policies from HR etc then try follow the same styling, titles, look and feel, then just keep it short and sweet. No it wont stay like that forever and you will find gaps that need closing but that’s good practice and why ISO27001 and other standards expect evolution and regular reviews.

Conclusion or rant over…

CHATGPT is a great tool and can be used by management to improve things and make your life easier but, its worth noting that with the large amount of content being published in live forums and corporate style social media that not everything you read is correct or right for you, or even written by a human with old school experience or AI (Analog Intelligence).

If your interesting in writing a follow up to this on your experiences please get in touch. Id love to hear your thoughts.

David Johnson

Virtual CISO and Cyber Consultant

*yes the image is an AI prompted design*